You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Documentation~/index.md
+72-70Lines changed: 72 additions & 70 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,30 +2,31 @@
2
2
3
3
The Unity Performance Testing Extension is a Unity Editor package that, when installed, provides an API and test case decorators to make it easier to take measurements/samples of Unity profiler markers, and other custom metrics outside of the profiler, within the Unity Editor and built players. It also collects configuration metadata, such as build and player settings, which is useful when comparing data against different hardware and configurations.
4
4
5
-
The Performance Testing Extension is intended to be used with, and complement, the Unity Test Runner framework.
5
+
The Performance Testing Extension is intended to be used with, and complement, the Unity Test Runner framework. For more information on how to create and run tests please refer to [Unity Test Runner documentation](https://docs.unity3d.com/Manual/testing-editortestsrunner.html).
6
+
6
7
7
8
**Important Note:** When tests are run with the Unity Test Runner, a development player is always built to support communication between the editor and player, effectively overriding the development build setting from the build settings UI or scripting API.
8
9
9
10
## Installing
10
11
11
12
To install the Performance Testing Extension package
12
13
1. Open the manifest.json file for your Unity project (located in the YourProject/Packages directory) in a text editor
13
-
2. Add com.unity.test-framework.performance to the dependencies as seen below
14
-
3. Add com.unity.test-framework.performance to the testables section. If there is not a testables section in your manifest.json file, go ahead and add it.
14
+
2. Add `com.unity.test-framework.performance` to the dependencies as seen below
15
+
3. Add `com.unity.test-framework.performance` to the testables section. If there is not a testables section in your manifest.json file, go ahead and add it.
15
16
4. Save the manifest.json file
16
17
5. Verify the Performance Testing Extension is now installed opening the Unity Package Manager window
17
-
6. Ensure you have created an Assembly Definition file in the same folder where your tests or scripts are that you’ll reference the Performance Testing Extension with. This Assembly Definition file needs to reference Unity.PerformanceTesting in order to use the Performance Testing Extension. Example of how to do this:
18
-
* Create a new folder for storing tests in ("Tests", for example)
19
-
* Create a new assembly definition file in the new folder using the context menu (right click/Create/Assembly definition) and name it "Tests" (or whatever you named the folder from step a. above)
20
-
* In inspector for the assembly definition file check "Test Assemblies", and then Apply.
21
-
* Open the assembly definition file in a text editor and add Unity.PerformanceTesting. To the references section. Save the file when you’re done doing this.
18
+
6. Ensure you have created an Assembly Definition file in the same folder where your tests or scripts are that you’ll reference the Performance Testing Extension with. This Assembly Definition file needs to reference `Unity.PerformanceTesting` in order to use the Performance Testing Extension. Example of how to do this:
19
+
1. Create a new folder for storing tests in ("Tests", for example)
20
+
2. Create a new assembly definition file in the new folder using the context menu (right click/Create/Assembly definition) and name it "Tests" (or whatever you named the folder from step a. above)
21
+
3. In inspector for the assembly definition file check "Test Assemblies", and then Apply.
22
+
4. Open the assembly definition file in a text editor and add Unity.PerformanceTesting. To the references section. Save the file when you’re done doing this.
@@ -40,7 +41,7 @@ To install the Performance Testing Extension package
40
41
```
41
42
42
43
43
-
>Example: assembly definition file
44
+
####Example: assembly definition file
44
45
45
46
```json
46
47
{
@@ -59,38 +60,34 @@ To install the Performance Testing Extension package
59
60
}
60
61
```
61
62
62
-
More information on how to create and run tests please refer to [Unity Test Runner docs](https://docs.unity3d.com/Manual/testing-editortestsrunner.html).
63
-
64
-
65
63
## Test Attributes
66
-
**[PerformanceTest]** - Non yeilding performance test.
64
+
**[PerformanceTest]** - Non-yielding performance test. This type of performance test starts and ends within the same frame.
-**aggregationType** : Preferred aggregation (default is median). Possible values are:
84
-
-Median, Average, Min, Max, Percentile
80
+
Median, Average, Min, Max, Percentile
85
81
-**percentile** : If aggregationType is Percentile, the percentile value used for the aggregation. e.g. 0.95.
86
82
-**increaseIsBetter** : Determines whether or not an increase in the measurement value should be considered a progression (performance improved) or a performance regression. Default is false. **NOTE:** This value is not used directly in the Performance Testing Extension, but recorded for later use in a reporting tool (such as the [Unity Performance Benchmark Reporter](https://github.com/Unity-Technologies/PerformanceBenchmarkReporter/wiki)) to determine whether or not a performance regression has occurred when used with a baseline result set.
87
83
-**threshold** : The threshold, as a percentage of the aggregated sample group value, to use for regression detection. Default value is 0.15f. **NOTE:** This value is not used directly in the Performance Testing Extension, but recorded for later use in a reporting tool (such as the [Unity Performance Benchmark Reporter](https://github.com/Unity-Technologies/PerformanceBenchmarkReporter/wiki)) to determine whether or not a performance regression has occurred when used with a baseline result set.
88
84
89
-
If unspecified a default SampleGroupDefinition will be used with the name of "Time", it is recommended to specify a name that is descriptive of what it is measuring.
90
85
91
86
## Taking measurements
92
87
93
-
The Performance Testing Extension provides several API methods you can use to take measurements in your performance test, depending on what you need to measure and how you want to do it. They are:
88
+
The Performance Testing Extension provides several API methods you can use to take measurements in your performance test, depending on what you need to measure and how you want to do it.
@@ -100,18 +97,16 @@ The Performance Testing Extension provides several API methods you can use to ta
100
97
101
98
The sections below detail the specifics of each measurement method with examples.
102
99
103
-
Preferred way is to use Measure.Method or Measure.Frames. They both do a couple of warmup iterations which are then used to decide how many iterations per measurement should be used.
104
100
105
-
106
-
**MethodMeasurement Method()**
101
+
### Measure.Method()
107
102
108
103
This will execute the provided method, sampling performance using the following additional properties/methods to control how the measurements are taken:
109
-
***WarmupCount(int n)** - number of times to to execute before measurements are collected. Default is 3 if not specified.
104
+
***WarmupCount(int n)** - number of times to to execute before measurements are collected. If unspecified, a default warmup is executed. This default warmup will wait for 7 ms. However, if less than 3 method executions have finished in that time, the warmup will wait until 3 method executions have completed.
110
105
***MeasurementCount(int n)** - number of measurements to capture. Default is 7 if not specified.
111
-
***IterationsPerMeasurement(int n)** - number of iterations per measurement to use
112
-
***GC()** - if specified, will measure the Gargage Collection allocation value.
106
+
***IterationsPerMeasurement(int n)** - number of method executions per measurement to use. If this value is not specified, the method will be executed as many times as possible until approximately 1 ms has elapsed.
107
+
***GC()** - if specified, will measure the total number of Garbage Collection allocation calls.
113
108
114
-
>Example 1: Simple method measurement using default values
109
+
####Example 1: Simple method measurement using default values
***ProfilerMarkers(...)** - sample profile markers per frame
136
+
Records time per frame by default and provides additional properties/methods to control how the measurements are taken:
137
+
***WarmupCount(int n)** - number of times to to execute before measurements are collected. If unspecified, a default warmup is executed. This default warmup will wait for 80 ms. However, if less than 3 full frames have rendered in that time, the warmup will wait until 3 full frames have been rendered.
138
+
***MeasurementCount(int n)** - number of frames to capture measurements. If this value is not specified, frames will be captured as many times as possible until approximately 500 ms has elapsed.
***ProfilerMarkers(...)** - sample profile markers per frame.
141
+
***Scope()** - measures frame times in a given asynchronous scope.
146
142
147
-
It will automatically select the number of warmup and runtime frames.
148
143
149
-
>Example 1: Simple frame time measurement
144
+
####Example 1: Simple frame time measurement using default values of at least 7 frames and default WarmupCount (see description above).
150
145
151
146
```csharp
152
147
[PerformanceUnityTest]
@@ -158,8 +153,9 @@ public IEnumerator Test()
158
153
}
159
154
```
160
155
161
-
In cases where you are measuring a system over frametime it is advised to disable frametime measurements and instead measure profiler markers for your system.
162
-
> Example 2: Sample profile markers per frame, disable frametime measurement
156
+
#### Example 2: Sample profile markers per frame, disable frametime measurement
157
+
158
+
If you’d like to sample profiler markers across multiple frames, and don’t have a need to record frametime, it is possible to disable the frame time measurement.
163
159
164
160
```csharp
165
161
[PerformanceUnityTest]
@@ -174,8 +170,22 @@ public IEnumerator Test()
174
170
}
175
171
```
176
172
173
+
#### Example 3: Sample frame times in a scope
174
+
175
+
```csharp
176
+
[PerformanceUnityTest]
177
+
publicIEnumeratorTest()
178
+
{
179
+
using (Measure.Frames().Scope())
180
+
{
181
+
yieldreturn...;
182
+
}
183
+
}
184
+
```
185
+
186
+
#### Example 3: Specify custom WarmupCount and MeasurementCount per frame
187
+
177
188
If you want more control, you can specify how many frames you want to measure.
178
-
> Example 3: Specify custom WarmupCount and MeasurementCount per frame
When you want to record samples outside of frame time, method time, or profiler markers, use a custom measurement. It can be any double value. A sample group definition is required.
223
+
Used to record profiler markers. Profiler marker timings will be sampled within the scope of the `using` statement. Note that deep and editor profiling markers are not available.
229
224
230
-
>Example 1: Use a custom measurement to capture total allocated memory
225
+
####Example 1: Use a custom measurement to capture total allocated memory
Records a custom sample. It can be any double value. A sample group definition is required.
249
+
When you want to record samples outside of frame time, method time, or profiler markers, use a custom measurement. It can be any double value. A sample group definition is required.
250
+
251
+
#### Example 1: Use a custom measurement to capture total allocated memory
255
252
256
253
```csharp
257
254
[PerformanceTest]
@@ -264,11 +261,15 @@ public void Test()
264
261
265
262
## Output
266
263
267
-
Each performance test will have a performance test summary. Every sample group will have multiple aggregated samples such as median, min, max, average, standard deviation, sample count, count of zero samples and sum of all samples.
264
+
When a test is selected in the Unity Test Runner window within the Unity Editor, each performance test will have a performance test summary. This summary includes every sample group’s aggregated samples such as median, min, max, average, standard deviation, sample count, count of zero samples and sum of all samples.
265
+
266
+
#### Example 1: Performance Test Summary from Unity Test Runner window
0 commit comments