Skip to content

Commit 9fc8213

Browse files
authored
Merge branch 'main' into contrib-update
2 parents a9c7894 + 748a0e9 commit 9fc8213

File tree

196 files changed

+1444
-3354
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

196 files changed

+1444
-3354
lines changed

docs/source/_templates/footer.html

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,10 +32,10 @@
3232
<p><a href="{{ pathto('tos/tos') }}">Terms of Service</a></p>
3333
</div>
3434
<div>
35-
<p><a class="external" href="https://www.firstinspires.org/about/privacy-policy">Privacy Policy</a></p>
35+
<p><a class="external" href="https://www.firstinspires.org/privacy-policy">Privacy Policy</a></p>
3636
</div>
3737
<div>
38-
<p><a href="https://www.firstinspires.org/report">Report an Incident</a></p>
38+
<p><a class="external" href="https://www.firstinspires.org/programs/youth-protection-program">Report an Incident</a></p>
3939
</div>
4040

4141
<div>
Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
AprilTag Challenges in DECODE presented by RTX
2+
==============================================
3+
4+
What are AprilTags?
5+
-------------------
6+
7+
Developed at the `University of Michigan
8+
<https://april.eecs.umich.edu/software/apriltag>`_, AprilTags are similar to a
9+
2D barcode or a simplified QR Code. It contains a numeric **ID code** and can
10+
be used for **location and orientation**.
11+
12+
In *FIRST* Tech Challenge during the DECODE presented by RTX season, AprilTags
13+
are used in three different ways:
14+
15+
1. On the OBELISK, the AprilTags are used to identify one of three MOTIFS that
16+
are randomized each MATCH.
17+
2. On the GOALS, AprilTags can be used to target the GOAL for teams to launch
18+
ARTIFACTS accurately into the correct GOAL.
19+
3. On the GOALS, AprilTags can be used as a visual odometry system, using the
20+
information that AprilTags can provide to calculate the position of the
21+
ROBOT on the FIELD (through a process called localization). See the
22+
:doc:`AprilTag Localization <../../vision_portal/apriltag_localization/apriltag-localization>`
23+
page for more information.
24+
25+
.. figure:: images/decode-apriltags.png
26+
:width: 50%
27+
:align: center
28+
:alt: Image showing the DECODE field and AprilTag locations
29+
30+
AprilTag IDs and Locations on the DECODE field.
31+
32+
AprilTags with Difficult Environmental Lighting
33+
-----------------------------------------------
34+
35+
One of the challenges teams will face this season is ensuring that their cameras
36+
are able to see the AprilTags correctly. AprilTags rely on the fact that the
37+
white and black portions of the AprilTags are contrasting colors - if the lighting
38+
in the environment doesn't allow for enough contrast, the AprilTag algorithm
39+
may not properly detect the AprilTag. Fortunately, there are things we can do
40+
with virtually every webcam that can help correct for environmental issues.
41+
42+
An excellent example situation came up in a warehouse. The DECODE field was set
43+
up in the warehouse, and using default settings with the
44+
``ConceptAprilTagEasy`` sample. When viewing the camera stream preview, the
45+
AprilTag on the OBELISK was completely washed out by sunlight striking the
46+
OBELISK directly on a sunny day, making the AprilTag unable to be seen. A
47+
different camera at a slightly different angle took another picture of the same
48+
scene, and the AprilTag can be seen but there is definitely too much direct light
49+
reflecting off the AprilTag, making it unable to be recognized. This scenario is
50+
very similar to a gymnasium where an event could be hosted, and on a sunny day
51+
light can interfere with a camera's ability to view an AprilTag. What can be done?
52+
53+
.. only:: html
54+
55+
.. grid:: 1 2 2 3
56+
:gutter: 2
57+
58+
.. grid-item-card::
59+
:class-header: sd-bg-dark font-weight-bold sd-text-white
60+
:class-body: sd-text-left body
61+
62+
Image #1 - Example
63+
64+
^^^
65+
66+
.. figure:: images/1-decode-washed-out-obelisk.*
67+
:align: center
68+
:width: 95%
69+
:alt: Image of DECODE field with obelisk AprilTag unable to be seen
70+
71+
+++
72+
73+
Washed Out AprilTag on OBELISK
74+
75+
.. grid-item-card::
76+
:class-header: sd-bg-dark font-weight-bold sd-text-white
77+
:class-body: sd-text-left body
78+
79+
Image #2 - Alternate View
80+
81+
^^^
82+
83+
.. figure:: images/2-decode-washed-out-obelisk.*
84+
:align: center
85+
:width: 85%
86+
:alt: Image of DECODE field from another perspective
87+
88+
+++
89+
90+
Alternate View of OBELISK
91+
92+
.. grid-item-card::
93+
:class-header: sd-bg-dark font-weight-bold sd-text-white
94+
:class-body: sd-text-left body
95+
96+
Image #3 - Alternate View
97+
98+
^^^
99+
100+
.. figure:: images/5-decode-warehouse-lighting.*
101+
:align: center
102+
:width: 85%
103+
:alt: Image showing light coming in from windows of warehouse
104+
105+
+++
106+
107+
Sunlight Entering Warehouse
108+
109+
.. only:: latex
110+
111+
.. list-table:: Different Views of Challenging Scenario
112+
:class: borderless
113+
114+
* - .. image:: images/1-decode-washed-out-obelisk.*
115+
- .. image:: images/2-decode-washed-out-obelisk.*
116+
- .. image:: images/5-decode-warehouse-lighting.*
117+
118+
The best way to counter this environmental lighting is to use the webcam
119+
settings within the SDK to adjust both the Gain and the Exposure settings at
120+
the same time. By simultaneously minimizing the exposure (lessening the amount of
121+
time light is allowed to strike the sensor each image frame) and maximizing
122+
the gain (amplifying the signal from the sensor) the resulting image will be
123+
darker than a normal image but elements of high contrast will be accentuated,
124+
like AprilTags, allowing them to be recognized. This can be experimented with
125+
using the ``ConceptAprilTagOptimizeExposure`` sample.
126+
127+
Sure enough, by minimizing the Exposure and maximizing the Gain of the webcam,
128+
the resulting images from the webcam were able to be used to recognize the
129+
problematic AprilTags. For more examples, the ``RobotAutoDriveToAprilTag...``
130+
sample OpModes also use this technique for adjusting the exposure and gain
131+
settings of the camera to ensure the AprilTags are readable under most
132+
conditions.
133+
134+
.. tip::
135+
One big advantage is that this technique (minimizing exposure while
136+
maximizing gain) is ALSO very popular in reducing motion blur for reading
137+
AprilTags while moving - so this has more than one benefit!
138+
139+
Here are examples of the images once the exposure and gain are set appropriately,
140+
one image has the AprilTag processing enabled to show that the AprilTag is
141+
being detected properly, and the other has processing disabled so that we can
142+
see the raw image being returned by the webcam.
143+
144+
.. only:: html
145+
146+
.. grid:: 1 2 2 2
147+
:gutter: 2
148+
149+
.. grid-item-card::
150+
:class-header: sd-bg-dark font-weight-bold sd-text-white
151+
:class-body: sd-text-left body
152+
153+
Image #4 - Processed Image
154+
155+
^^^
156+
157+
.. figure:: images/3-decode-recognized-obelisk.*
158+
:align: center
159+
:width: 95%
160+
:alt: Image of DECODE field with obelisk AprilTag being processed
161+
162+
+++
163+
164+
Processed Image showing Detections
165+
166+
.. grid-item-card::
167+
:class-header: sd-bg-dark font-weight-bold sd-text-white
168+
:class-body: sd-text-left body
169+
170+
Image #5 - Raw Processed Image
171+
172+
^^^
173+
174+
.. figure:: images/4-decode-recognized-obelisk-raw.*
175+
:align: center
176+
:width: 95%
177+
:alt: Image of raw processed DECODE field
178+
179+
+++
180+
181+
Image without AprilTag processing
182+
183+
.. only:: latex
184+
185+
.. list-table:: Resulting Images
186+
:class: borderless
187+
188+
* - .. image:: images/3-decode-recognized-obelisk.*
189+
- .. image:: images/4-decode-recognized-obelisk-raw.*
190+
191+
Reading Multiple AprilTags on the OBELISK
192+
-----------------------------------------
193+
194+
The OBELISK is an equilateral triangular prism (we know, real obelisks have 4
195+
sides) which is positioned with 1 of the rectangular faces centered on the
196+
GOAL-side of the FIELD, just outside of the FIELD perimeter. When ROBOTS are
197+
set up on the field contacting their ALLIANCE'S GOAL, it is a very real
198+
possibility that the ROBOT's camera will see and process multiple AprilTags.
199+
200+
.. warning::
201+
It might seem logical to read both AprilTags and use those two tags to
202+
determine (and verify) which AprilTag is actually being seen. However, there
203+
is no defined order for AprilTags on an OBELISK, so this is not reliable.
204+
205+
206+
.. figure:: images/6-decode-obelisk-tags.*
207+
:align: center
208+
:width: 75%
209+
:alt: Image showing OBELISK with more than one AprilTag visible
210+
211+
View of AprilTags on OBELISK from BLUE GOAL
212+
213+
A reliable way to determine which AprilTag is truly showing on the FIELD
214+
is to move the ROBOT into a position where the AprilTag on the front face of
215+
the OBELISK is the only tag that can be viewed.
216+
217+
Good Luck this season!
327 KB
Loading
457 KB
Loading
212 KB
Loading
224 KB
Loading
401 KB
Loading
80.8 KB
Loading
166 KB
Loading

docs/source/apriltag/vision_portal/vision_processor_init/vision-processor-init.rst

Lines changed: 10 additions & 101 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ Vision Processor Initialization
44
Processor Initialization - Overview
55
-----------------------------------
66

7-
Your OpMode must **first prepare** for using AprilTag and/or TensorFlow Object
8-
Detection (TFOD) commands, or methods.
7+
Your OpMode must **first prepare** for using AprilTag and/or
8+
Color Processing commands, or methods.
99

1010
In the INIT portion of your Java or Blocks code, before ``waitForStart()``, use
1111
these steps:
@@ -18,11 +18,11 @@ these steps:
1818

1919
- **Step 2.** *Required:*
2020

21-
- Create the **AprilTag Processor** (or the **TFOD
21+
- Create the **AprilTag Processor** (or the **Color Locator or Color Sensor
2222
Processor**), to analyze frames streaming in from the camera. "Under the
2323
hood", the Apriltag Processor is attached to an EOCV **pipeline**, which
2424
performs various steps, in order, to each stream frame. The stream is the
25-
input to the pipeline. A similar process happens for TFOD.
25+
input to the pipeline. A similar process happens for Color Processing.
2626

2727
- **Step 3.** *Required:*
2828

@@ -141,106 +141,15 @@ Here the object ``myAprilTagProcessorBuilder`` was not created; the build was
141141
performed directly on ``myAprilTagProcessor``. The Builder pattern allows the
142142
"dot" methods to be chained in a single Java statement ending with ``.build()``.
143143

144-
TensorFlow Initialization - Easy
145-
--------------------------------
144+
Color Processing Initialization
145+
-------------------------------
146146

147-
**Step 2** is similar for creating the **TensorFlow TFOD Processor**, software
147+
**Step 2** is similar for creating the **Color Sensor/Locator Processor**, software
148148
that evaluates frames streaming in from the camera.
149149

150-
The SDK provides an "easy" way to create the processor, using only **defaults**
151-
and not mentioning a "Builder":
152-
153-
.. tab-set::
154-
.. tab-item:: Blocks
155-
:sync: blocks
156-
157-
.. figure:: images/040-TFOD-Processor-easy.png
158-
:width: 75%
159-
:align: center
160-
:alt: Easy TFOD Initialization
161-
162-
Easy TensorFlow TFOD Processor Initialization without a Builder
163-
164-
.. tab-item:: Java
165-
:sync: java
166-
167-
Example of TensorFlow TFOD Processor Initialization without a Builder
168-
169-
.. code-block:: java
170-
171-
TfodProcessor myTfodProcessor;
172-
// Create the TensorFlow Object Detection processor and assign it to a variable.
173-
myTfodProcessor = TfodProcessor.easyCreateWithDefaults();
174-
175-
176-
TensorFlow Initialization - Builder
177-
-----------------------------------
178-
179-
The SDK also provides the "Builder" way to create the processor, allowing
180-
**custom settings**.
181-
182-
**Builder** is a Java pattern or structure for adding features or parameters,
183-
finalized with the ``.build()`` command. Such features are **not** modified
184-
later during an OpMode.
185-
186-
*Inside the SDK, even the "easy" process uses the Builder pattern to set the default parameters.*
187-
188-
.. tab-set::
189-
.. tab-item:: Blocks
190-
:sync: blocks
191-
192-
.. figure:: images/050-TFOD-Processor-builder.png
193-
:width: 75%
194-
:align: center
195-
:alt: TFOD Initialization with Builder
196-
197-
TensorFlow TFOD Processor Initialization with a Builder
198-
199-
.. tab-item:: Java
200-
:sync: java
201-
202-
.. code-block:: java
203-
204-
TfodProcessor.Builder myTfodProcessorBuilder;
205-
TfodProcessor myTfodProcessor;
206-
207-
// Create a new TFOD Processor Builder object.
208-
myTfodProcessorBuilder = new TfodProcessor.Builder();
209-
210-
// Optional: set other custom features of the TFOD Processor (4 are shown here).
211-
myTfodProcessorBuilder.setMaxNumRecognitions(10); // Max. number of recognitions the network will return
212-
myTfodProcessorBuilder.setUseObjectTracker(true); // Whether to use the object tracker
213-
myTfodProcessorBuilder.setTrackerMaxOverlap((float) 0.2); // Max. % of box overlapped by another box at recognition time
214-
myTfodProcessorBuilder.setTrackerMinSize(16); // Min. size of object that the object tracker will track
215-
216-
// Create a TFOD Processor by calling build()
217-
myTfodProcessor = myTfodProcessorBuilder.build();
218-
219-
This example shows only 4 TFOD Processor Builder features; others are
220-
available. Most others relate to custom TFOD Models, beyond this scope of this
221-
VisionPortal Guide.
222-
223-
TensorFlow Java Builder Chain
224-
-----------------------------
225-
226-
The Builder pattern can be implemented in a streamlined manner, using Java. The following code is equivalent to the above individual method calls.
227-
228-
Comments are omitted here, to clearly illustrate the chaining.
229-
230-
.. code-block:: java
231-
232-
TfodProcessor myTfodProcessor;
233-
234-
myTfodProcessor = new TfodProcessor.Builder()
235-
.setMaxNumRecognitions(10)
236-
.setUseObjectTracker(true)
237-
.setTrackerMaxOverlap((float) 0.2)
238-
.setTrackerMinSize(16)
239-
.build();
240-
241-
Here the object ``myTfodProcessorBuilder`` was not created; the build was
242-
performed directly on ``myTfodProcessor``. The Builder pattern allows the "dot"
243-
methods to be chained in a single Java statement ending with ``.build()``.
150+
The Builder is going to look a little different depending on if you are using
151+
the **Color Sensor** or the **Color Locator**. For more information, see the
152+
:doc:`Color Processing Documentation </color_processing/index>`.
244153

245154
Enabling and Disabling Processors
246155
---------------------------------

0 commit comments

Comments
 (0)