Skip to content

Commit 17721a0

Browse files
committed
more polish
1 parent bdcf957 commit 17721a0

File tree

1 file changed

+16
-18
lines changed
  • com.unity.perception/Documentation~/FAQ

1 file changed

+16
-18
lines changed

com.unity.perception/Documentation~/FAQ/FAQ.md

Lines changed: 16 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ This Randomizer takes a list of `PrefabCluster` assets, then, on each Iteration,
148148
<details>
149149
<summary><strong>Q: How can I specify an exact number of objects to place using the sample foreground object placement Randomizer?</strong> </summary><br>
150150

151-
The provided `ForegroundObjectPlacementRandomizer` uses Poisson Disk sampling to find randomly positioned points in the space denoted by the provided `Width` and `Height` values. The distance between the sampled points will be at equal to `Separation Distance`. The number of sampled points will be the maximum number of points in the given area that match these criteria.
151+
The provided `ForegroundObjectPlacementRandomizer` uses Poisson Disk sampling to find randomly positioned points in the space denoted by the provided `Width` and `Height` values. The lower bound on the distance between the sampled points will be `Separation Distance`. The number of sampled points will be the maximum number of points in the given area that match these criteria.
152152

153153
Thus, to limit the number of spawned objects, you can simply introduce a hard limit in the `for` loop that iterates over the Poisson Disk samples, to break out of the loop if the limit is reached. For example:
154154

@@ -192,7 +192,7 @@ This will guarantee an upper limit of 50 on the number of objects. To have exact
192192

193193
There are a number of ways for procedurally placing objects while avoiding any overlap between them, and most of these methods can be rather complex and need to place objects in a sequence. All the modifications to the objects (like scale, rotation, etc.) would also need to happen before the next object is placed, so that the state of the world is fully known before each placement.
194194

195-
Here, we are going to introduce a rather simple modification in the sample foreground placement code provided with the package. In each Iteration, a random scale factor is chosen, and then a desirable separation distance is calculated based on this scale factor and the list of given prefabs. We scale the objects here to introduce additional randomization, and the fact that once we have placed the objects we can no longer scale them.
195+
Here, we are going to introduce a rather simple modification in the sample foreground placement code provided with the package. In each Iteration, a random scale factor is chosen, and then a desirable separation distance is calculated based on this scale factor and the list of given prefabs. We also scale the objects here to introduce additional randomization, due to the fact that once we have placed the objects we can no longer scale them.
196196

197197
Based on the value given for `Non Overlap Guarantee`, this Randomizer can either reduce the amount of overlap or completely remove overlap.
198198

@@ -324,7 +324,7 @@ public class NoOverlapForegroundObjectPlacementRandomizer : Randomizer
324324
</details>
325325

326326
<details>
327-
<summary><strong>Q: What if I don't want randomized object placement? Can I move my objects in a non-randomized predefined manner on each frame?</strong> </summary><br>
327+
<summary><strong>Q: What if I don't want randomized object placement? Can I move my objects in a non-randomized deterministic manner on each frame?</strong> </summary><br>
328328

329329
Even though we call them Randomizers, you can use a Randomizer to perform any task through-out the execution lifecycle of your Scenario. The power of the Randomizers comes from the lifecycle hooks that they have into the Iteration and the Scenario, making it easy to know and guarantee when and in which order in the life of your simulation each piece of code runs. These functions include:
330330
* `OnEnable`
@@ -356,7 +356,7 @@ protected override void OnIterationStart()
356356

357357
The objects instantiated by the sample foreground Randomizer are all parented to an object named `Foreground Objects` at the root of the Scene Hierarchy. To modify the orientation of the objects, you can simply rotate this parent object at the beginning of the Scenario.
358358

359-
Alternatively, you could also place `Foreground Objects` inside another GameObject in the Scene using the `Transform.SetParent()` method, and then modifying the local position and and rotation of `Foreground Objects` in such a way that makes the objects appear on the surface of the parent GameObject.
359+
Alternatively, you could also place `Foreground Objects` inside another GameObject in the Scene using the `Transform.SetParent()` method, and then modify the local position and rotation of `Foreground Objects` in such a way that makes the objects appear on the surface of the parent GameObject.
360360

361361
To achieve more natural placement, you could also use Unity's physics engine to drop the objects on a surface, let them settle, and then capture an image. To achieve this, you would just need to have sufficient frames in each Iteration of the Scenario (instead of the default 1 frame per iteration), and set your Perception Camera's capture interval to a large enough number that would make it capture each Iteration once after the objects have settled. This example is explained in more detail in the [Perception Camera](#perception-camera) section of this FAQ.
362362

@@ -378,7 +378,7 @@ To achieve more natural placement, you could also use Unity's physics engine to
378378
## <a name="perception-camera">Perception Camera</a>
379379

380380
<details>
381-
<summary><strong>Q: What is the relationship between the Scenario's lifecycle properties (Iterations and Frames per Iteration) in conjunction with the Perception Camera's timing properties (Simulation Delta Time, Start at Frame, and Frames Between Captures)?</strong> </summary><br>
381+
<summary><strong>Q: What is the relationship between the Scenario's lifecycle properties (Iterations and Frames per Iteration) and the Perception Camera's timing properties (Simulation Delta Time, Start at Frame, and Frames Between Captures)?</strong> </summary><br>
382382

383383
Each Iteration of the Scenario resets the Perception Camera's timing variables. Thus, you can think of each Iteration of the Scenario as one separate Perception Camera sequence, in which the camera's internal timing properties come into play. For instance, if you have 10 `Frames Per Iteration` on your Scenario, and your Perception Camera's `Start at Frame` value is set to 8, you will get two captures from the camera at the 9th and 10th frames of each Iteration (note that `Start at Frame` starts from 0). Similarly, you can use the `Frames Between Captures` to introduce intervals between captures. A value of 0 leads to all frames being captured.
384384

@@ -390,22 +390,20 @@ Each Iteration of the Scenario resets the Perception Camera's timing variables.
390390

391391
The Perception Camera can be set to capture at specific frame intervals, rather than every frame. The `Frames Between Captures` value is set to 0 by default, which causes the camera to capture all frames; however, you can change this to 1 to capture every other frame, or larger numbers to allow more time between captures. You can also have the camera start capturing at a certain frame rather the first frame, by setting the `Start at Frame` value to a value other than 0. All of this timing happens within each Iteration of the Scenario, and gets reset when you advance to the next Iteration. Therefore, the combination of these properties and the Scenario's `Frames Per Iteration` property allows you to randomize the state of your Scene at the start of each Iteration, let things run for a number of frames, and then capture the Scene at the end of the Iteration.
392392

393-
Suppose we need to drop a few objects into the Scene, let them interact physically and settle after a number of frames, and then capture their final state once. Afterwards, we want to repeat this cycle by randomizing the initial positions of the objects, dropping them, and capturing the final state again. We will set the Scenario's `Frames Per Iteration` to 300, which should be sufficient for the objects to get close to a settled position (this depends on the value you use for `Simulation Delta Time` in Perception Camera and the physical properties of the engine and objects, and can be found through experimentation). We also set the `Start at Frame` value of the Perception Camera to 290, and the `Frames Between Captures` to a sufficiently large number (like 100), so that we only get one capture per Iteration of the Scenario. The results look like below:
393+
Suppose we need to drop a few objects into the Scene, let them interact physically and settle after a number of frames, and then capture their final state once. Afterwards, we want to repeat this cycle by randomizing the initial positions of the objects, dropping them, and capturing the final state again. We will set the Scenario's `Frames Per Iteration` to 300, which should be sufficient for the objects to get close to a settled position (this depends on the value you use for `Simulation Delta Time` in Perception Camera and the physical properties of the engine and objects, and can be found through experimentation). We also set the `Start at Frame` value of the Perception Camera to 290, and the `Frames Between Captures` to a sufficiently large number (like 100), so that we only get one capture per Iteration of the Scenario. The results look like below. Note how the bounding boxes only update after the objects are fairly settled. These are the timestamps at which captures are happening.
394394

395395

396396
<p align="center">
397397
<img src="images/object_drop.gif" width="700"/>
398398
</p>
399399

400-
Note how the bounding boxes only update after the objects are fairly settled. These are the points at which captures are happening.
401-
402400
---
403401
</details>
404402

405403
<details>
406-
<summary><strong>Q: I do not want to use the Perception Camera to control the timing of my simulation or capture on a scheduled basis. Can I have a Perception Camera in my Scene and trigger captures manually from other scripts?</strong></summary><br>
404+
<summary><strong>Q: I do not want Perception Camera to control the delta time of my simulation or capture on a scheduled basis. Can I have a Perception Camera in my Scene that does not affect timings and trigger captures manually from other scripts?</strong></summary><br>
407405

408-
Yes. The Perception Camera offers two trigger modes, `Scheduled` and `Manual`, and these can be chosen in the editor UI for the camera. If you select the `Manual` mode, you will be able to trigger captures by calling the `RequestCapture()` method of `PerceptionCamera`. In this mode, you still have an option to dictate your simulation delta time with this camera, in order to have deterministic simulation progress between rendered frames. This is controlled using the `Affect Simulation Timing` checkbox.
406+
Yes. The Perception Camera offers two trigger modes, `Scheduled` and `Manual`, and these can be chosen in the editor UI for the Perception Camera component. If you select the `Manual` mode, you will be able to trigger captures by calling the `RequestCapture()` method of `PerceptionCamera`. In this mode, you have the option to not have this camera dictate your simulation delta time. This is controlled using the `Affect Simulation Timing` checkbox.
409407

410408
---
411409
</details>
@@ -425,7 +423,7 @@ However, you can have more than one Perception Camera in the Scene, if only one
425423
<summary><strong>Q: My RGB images look darker than what I see in Unity Editor, when I render the Perception Camera to a texture. How can I fix this?</strong>
426424
</summary><br>
427425

428-
This issue is caused by the color format of the texture. In the ***Inspector** view of the render texture, set color format to `R8G8B8A8_SRGB`.
426+
This issue is caused by the color format of the texture. In the ***Inspector*** view of the render texture, set color format to `R8G8B8A8_SRGB`.
429427

430428
---
431429
</details>
@@ -445,7 +443,7 @@ This can be done by adding custom annotations to your dataset. Have a look at [t
445443
<summary><strong>Q: Objects in my captured images have jagged edges, how can I fix this?</strong>
446444
</summary><br>
447445

448-
This is a common issue with rendering graphics into pixel grids (digital images), when the resolution of the grid is not high enough to perfectly display the slanting lines in the image. The common solution to this issue is the use of anti-aliasing methods, and Unity offers a number of these in both URP and HDRP. To experiment with anti-aliasing, go to the ***Inspector*** view of your Perception Camera object and in the Camera component, change `Anti-aliasing` from `None` to another option.
446+
This is a common issue with rendering graphics into pixel grids (digital images), when the resolution of the grid is not high enough to perfectly display slanting lines. The common solution to this issue is the use of anti-aliasing methods, and Unity offers a number of these in both URP and HDRP. To experiment with anti-aliasing, go to the ***Inspector*** view of your Perception Camera object and in the Camera component, change `Anti-aliasing` from `None` to another option.
449447

450448
---
451449
</details>
@@ -454,7 +452,7 @@ This is a common issue with rendering graphics into pixel grids (digital images)
454452
<summary><strong>Q: I am using an HDRP Unity project with Perception and my images have significant blurring around most objects. How can I remove this blur?</strong>
455453
</summary><br>
456454

457-
The effect you are observing here is motion blur, which is happens because the placement Randomizers used in the Perception tutorial cache their instantiated objects from one frame to the next, and move them to new locations on each frame instead of destroying them and creating new ones. This "motion" of the objects causes the motion blur effect to kick in.
455+
The effect you are observing here is motion blur, which happens because the placement Randomizers used in the Perception tutorial cache their instantiated objects from one Iteration to the next, and move them to new locations on each Iteration instead of destroying them and creating new ones. This "motion" of the objects causes the motion blur effect to kick in.
458456

459457

460458
HDRP projects have motion blur and a number of other post processing effects enabled by default. To disable motion blur or any other effect, follow these steps:
@@ -471,15 +469,15 @@ HDRP projects have motion blur and a number of other post processing effects ena
471469
</details>
472470

473471
<details>
474-
<summary><strong>Q: Are all post processing effects provided by Unity safe to use?</strong>
472+
<summary><strong>Q: Are all post processing effects provided by Unity safe to use in synthetic datasets?</strong>
475473
</summary><br>
476474

477-
A couple of important notes to keep in mind with post-processing revolve around randomness:
475+
No. When using post processing effects, you need to be careful about issues regarding randomness and determinism:
478476

479-
* Some effects need to be Randomized from frame to frame, e.g. Film Grain). The Film Grain effect provided by Unity is not sufficiently randomized for model training and can thus mislead your CV model in the training process.
480-
* Even if an effect is properly randomized, using it would make your overall randomization strategy non-deterministic, as it would use random number generators outside of the Perception package provided Samplers.
477+
* There are certain post processing effects that need randomization internally, e.g. film grain. The film grain effect provided by Unity is not sufficiently randomized for model training and can thus mislead your CV model to look for a specific noise pattern during prediction.
478+
* Even if such an effect is properly randomized, using a randomized effect would make your overall randomization strategy non-deterministic, meaning you would not be able to reproduce your datasets. This is because the effect would internally use random number generators outside of the Perception package provided Samplers. If you have access to the source code for a randomized effect, you can modify it to only use Perception Samplers for random number generation, which would make its behavior deterministic and thus appropriate for use in synthetic datasets that need to be reproducible.
481479

482-
To make sure you do not run into insufficient randomization or non-determinism, it would be best to implement effects such as Film Grain yourself and only use the Perception provided Samplers and Parameters in order to guarantee determinism.
480+
To make sure you do not run into insufficient randomization or non-determinism, it would be best to implement effects such as film grain yourself or modify existing code to make sure no random number generators are used except for the Samplers provided in the Perception package.
483481

484482
---
485483
</details>

0 commit comments

Comments
 (0)