Skip to content

Commit a2df550

Browse files
committed
Update the primary vision docs page for the new API page
1 parent b7abd9c commit a2df550

File tree

3 files changed

+102
-144
lines changed

3 files changed

+102
-144
lines changed

images/content/vision/spherical.png

41.2 KB
Loading
29.2 KB
Loading

programming/sr/vision/index.md

Lines changed: 102 additions & 144 deletions
Original file line numberDiff line numberDiff line change
@@ -10,85 +10,69 @@ Vision
1010
This documentation refers to a feature which is only available on the physical robot kits.
1111
</div>
1212

13-
The `sr.robot` library contains support for detecting libkoki markers with the provided webcam.
13+
The `sr.robot` library contains support for detecting fiducial markers with the provided webcam.
1414
Markers are attached to various items in the Student Robotics arena.
1515
Each marker encodes a number in a machine-readable way, which means that robots can identify these objects.
16-
For information on which markers codes are which, see the [markers page](/docs/programming/sr/vision/markers).
16+
For information on which markers codes are which, see the [markers page](./markers).
1717

1818
Using knowledge of the physical size of the different markers and the characteristics of the webcam,
19-
libkoki can calculate the position of markers in 3D space relative to the camera.
19+
your robot can calculate the position of markers in 3D space relative to the camera.
2020
Therefore, if the robot can see a marker that is at a fixed location in the arena,
2121
a robot can calculate its exact position in the arena.
2222

23-
The `sr.robot` library provides all of this power through a single function, `R.see`:
23+
Under the hood, the vision system is based on [Zoloto](https://zoloto.readthedocs.io/), a wrapper around [OpenCV's ArUco library](https://docs.opencv.org/4.5.4/d5/dae/tutorial_aruco_detection.html), using the `36H11` marker set from [April](https://april.eecs.umich.edu/software/apriltag)
2424

25-
~~~~~ python
26-
from sr.robot import *
27-
R = Robot()
28-
markers = R.see()
29-
~~~~~
25+
[Camera](#camera) {#camera}
26+
===========================
27+
28+
The interface to the vision system is through the camera, accessible through `r.camera`.
3029

31-
When called, this function takes a photo through the webcam and searches for markers within it.
32-
It returns a list of `Marker` objects, each of which describes one of the markers that were found in the image.
33-
A detailed description of the attributes of Marker objects is provided [later in this page](#Marker).
30+
see
31+
: Take a photo through the webcam, and return a list of [`Marker`](#Marker) instances, each of which describes one of the markers that were found in the image.
3432

3533
Here's an example that will repeatedly print out the distance to each arena marker that the robot can see:
3634

3735
~~~~~ python
38-
from sr.robot import *
36+
from sr.robot3 import *
3937
R = Robot()
4038

4139
while True:
42-
markers = R.see()
40+
markers = R.camera.see()
4341
print("I can see", len(markers), "markers:")
4442

4543
for m in markers:
46-
if m.info.marker_type == MARKER_ARENA:
47-
print(" - Marker #{0} is {1} metres away".format(m.info.code, m.dist))
44+
print(" - Marker #{0} is {1} metres away".format(m.id, m.distance))
4845
~~~~~
4946

50-
[Choosing Resolution](#ChoosingResolution) {#ChoosingResolution}
51-
-------------------
47+
see_ids
48+
: Take a photo through the webcam, and return a list of marker ids (**not** full `Marker` objects). This doesn't do the same orientation calculations as `see`, and so is much faster to run.
49+
50+
~~~~~ python
51+
from sr.robot3 import *
52+
R = Robot()
53+
54+
while True:
55+
marker_ids = R.camera.see_ids()
5256

53-
By default, the `R.see` function will take a photo at a resolution of 800x600.
54-
The resolution that this image is taken at can be changed using the optional `res` argument:
57+
if 0 in marker_ids:
58+
print("I can see marker 0!")
59+
else:
60+
print("I cannot see marker 0!")
61+
~~~~~
62+
63+
save
64+
: Take a photo through the webcam, and save it to the provided location.
5565

5666
~~~~~ python
57-
# Take a photo at 1280 x 1024
58-
markers = R.see(res=(1280, 1024))
67+
from sr.robot3 import *
68+
R = Robot()
69+
70+
# `R.usbkey` is the path to your USB drive
71+
marker_ids = R.camera.save(R.usbkey / "initial-view.png")
5972
~~~~~
6073

61-
There are currently two kinds of webcam issued with SR kit: the Logitech C500 and C270.
62-
They support the following resolutions:
63-
64-
| Resolution | C500 | C270 |
65-
|---------------------------|-------|-------|
66-
| 160 x 120 | yes | yes |
67-
| 176 x 144 | yes | yes |
68-
| 320 x 176 | | yes |
69-
| 320 x 240 | yes | yes |
70-
| 352 x 288 | yes | yes |
71-
| 432 x 240 | | yes |
72-
| 544 x 288 | | yes |
73-
| 640 x 360 | yes | |
74-
| 640 x 400 | yes | |
75-
| 640 x 480 | yes | yes |
76-
| 752 x 416 | | yes |
77-
| 800 x 448 | | yes |
78-
| **800 x 600** (Default) | yes | yes |
79-
| 864 x 480 | | yes |
80-
| 960 x 544 | | yes |
81-
| 960 x 720 | yes | yes |
82-
| 1024 x 576 | | yes |
83-
| 1280 x 720 | yes | yes |
84-
| 1280 x 800 | yes | |
85-
| 1280 x 960 | | yes |
86-
| 1280 x 1024 | yes | |
87-
88-
There are advantages and disadvantages to switching resolution.
89-
Smaller images will process faster, but markers will be less likely to be detected within them.
90-
Additionally, the act of changing resolution takes a significant amount of time.
91-
The optimum resolution to use in a given situation is best determined through experiment.
74+
[Field of View](#fov) {#fov}
75+
-------------------
9276

9377
The Logitech C500 has a [field of view][fov] of 72&deg; and the C270 has a field of view of 60&deg;.
9478

@@ -101,12 +85,6 @@ The vision system describes the markers it can see using three coordinate
10185
systems. These are intended to be complementary to each other and contain
10286
the same information in different forms.
10387

104-
The individual coordinate systems used are detailed below on the
105-
[`Point`](#Point) object, which represents a point in space.
106-
Both it and the [`Orientation`](#Orientation) object provide further
107-
details about what measurements of rotation or position mean for their
108-
attributes.
109-
11088
The axis definitions match those in common use, as follows:
11189

11290
x-axis
@@ -132,129 +110,109 @@ for that in your usage of the vision system's data.
132110
[Objects of the Vision System](#vision_objects) {#vision_objects}
133111
==============================
134112

113+
The vision system is made up of a number of objects, the primary of which is the `Marker`:
114+
135115
[`Marker`](#Marker) {#Marker}
136116
----------
137117
A `Marker` object contains information about a *detected* marker.
138118
It has the following attributes:
139119

140-
info
141-
: A [`MarkerInfo`](#MarkerInfo) object containing information about the type of marker that was detected.
120+
id
121+
: The id of the marker.
142122

143-
centre
144-
: A [`Point`](#Point) describing the position of the centre of the marker.
123+
size
124+
: The physical size of the marker, as the vision system expects it.
145125

146-
vertices
147-
: A list of 4 [`Point`](#Point) instances, each representing the position of the black corners of the marker.
126+
pixel_centre
127+
: A [`Coordinate`](#Coordinate) describing the position of the centre of the marker.
148128

149-
dist
150-
: An alias for `centre.polar.length`
129+
pixel_corners
130+
: A list of 4 [`Coordinate`](#Coordinate) instances, each representing the position of the corners of the marker.
151131

152-
rot_y
153-
: An alias for `centre.polar.rot_y`
132+
distance
133+
: The distance between the camera and the centre of the marker, in metres.
154134

155135
orientation
156136
: An [`Orientation`](#Orientation) instance describing the orientation of the marker.
157137

158-
res
159-
: The resolution of the image that was taken from the webcam.
160-
A 2-item tuple: (width, height).
138+
spherical
139+
: A [`Spherical`](#Spherical) instance describing the position relative to the camera.
140+
141+
cartesian
142+
: A [`ThreeDCoordinates`][#ThreeDCoordinates] instance describing the absolute position of the marker relative to the camera.
161143

162-
timestamp
163-
: The timestamp at which the image was taken (a float).
144+
[`Coordinate`](#Coordinate) {#Coordinate}
145+
---------
164146

165-
[`MarkerInfo`](#MarkerInfo) {#MarkerInfo}
166-
--------------
147+
A `Coordinate` object contains an `x` and `y` attribute. The exact meaning and unit of these attributes depends on its source.
167148

168-
The `MarkerInfo` object contains information about a marker.
169-
It has the following attributes:
149+
[`ThreeDCoordinate`](#ThreeDCoordinate) {#ThreeDCoordinate}
150+
---------
170151

171-
code
172-
: The numeric code of the marker.
152+
A `ThreeDCoordinate` object contains an `x`, `y` and `z` attribute. The exact meaning and unit of these attributes depends on its source.
173153

174-
marker_type
175-
: The type of object that this marker represents.<br />
176-
One of:
154+
[`Orientation`](#Orientation) {#Orientation}
155+
---------------
177156

178-
* `MARKER_ARENA`
179-
<!-- Add other game-specific values here too -->
157+
An `Orientation` object describes the orientation of a marker.
180158

181-
offset
182-
: The offset of the numeric code of the marker from the lowest numbered marker of its type.
183-
For example: markers 28 and 29, which are the lowest numbered markers that represent robots, have offsets of 0 and 1 respectively.
159+
![]({{ site.baseurl }}/images/content/vision/yawpitchroll.png)
184160

185-
size
186-
: The size of the marker in metres.
187-
This is the length of the side of the main black body of the marker.
161+
pitch
162+
: Rotation of the marker about the cartesian x-axis, in radians.
188163

189-
[`Point`](#Point) {#Point}
190-
---------
164+
Leaning a marker away from the camera increases the value of `rot_x`, while
165+
leaning it towards the camera decreases it. A value of 0 indicates that the
166+
marker is upright.
191167

192-
A `Point` object describes a position in three different ways.
193-
These are accessed through the following attributes:
168+
yaw
169+
: Rotation of the marker about the cartesian y-axis, in radians.
194170

195-
image
196-
: The pixel coordinates of the point in the image, with the origin (0,0) in the top-left of the image.
197-
This has two attributes: `x` and `y`.
171+
Turning a marker clockwise (as viewed from above) increases the value of
172+
`rot_y`, while turning it anticlockwise decreases it. A value of 0 means
173+
that the marker is perpendicular to the line of sight of the camera.
198174

199-
world
200-
: The [Cartesian coordinates](https://en.wikipedia.org/wiki/Cartesian_coordinate_system) of the point in 3D space.
201-
This has three attributes: `x`, `y`, and `z`, each of which specifies a distance in metres.
202-
Positions in front of, to the right, or above the camera are positive.
203-
Positions to the left or below are negative.
175+
roll
176+
: Rotation of the marker about the cartesian z-axis, in radians.
204177

205-
polar
206-
: The [polar coordinates](https://en.wikipedia.org/wiki/Polar_coordinate_system) of the point in 3D space.<br />
207-
This has three attributes:
178+
Turning a marker anticlockwise (as viewed from the camera) increases the
179+
value of `rot_z`, while turning it clockwise decreases it. A value of 0
180+
indicates that the marker is upright.
208181

209-
length
210-
: The distance to the point.
182+
rot_x
183+
: An alias for `pitch`.
211184

212-
rot_x
213-
: Rotation about the x-axis in degrees.
214-
Positions above the camera are positive.
185+
rot_y
186+
: An alias for `yaw`.
215187

216-
rot_y
217-
: Rotation about the y-axis in degrees.
218-
Positions to the right of the camera are positive.
188+
rot_z
189+
: An alias for `roll`.
219190

220-
For example, the following code displays the polar coordinate of a `Point` object `p`:
191+
rotation_matrix
192+
: The rotation matrix represented by this orientation. A list of 3 lists, each with 3 items.
221193

222-
~~~~~ python
223-
print("length", p.polar.length)
224-
print("rot_x", p.polar.rot_x)
225-
print("rot_y", p.polar.rot_y)
226-
~~~~~
194+
quaternion
195+
: The [Quarternion](https://kieranwynn.github.io/pyquaternion/#quaternion-features) instance represented by this orientation.
227196

228-
[`Orientation`](#Orientation) {#Orientation}
197+
[`Spherical`](#Spherical) {#Spherical}
229198
---------------
230199

231-
An `Orientation` object describes the orientation of a marker. It has three attributes:
200+
The spherical coordinates system has three values to specify a specific point in space.
232201

233-
rot_x
234-
: Rotation of the marker about the x-axis.
202+
![]({{ site.baseurl }}/images/content/vision/spherical.png)
235203

236-
Leaning a marker away from the camera increases the value of `rot_x`, while
237-
leaning it towards the camera decreases it. A value of 0 indicates that the
238-
marker is upright.
239204

240-
rot_y
241-
: Rotation of the marker about the y-axis.
205+
- r - The radial distance, the distance from the origin to the point, in metres.
206+
- θ (theta) - The angle from the azimuth to the point, in radians.
207+
- φ (phi) - The polar angle from the plane of the camera to the point, in radians.
242208

243-
Turning a marker clockwise (as viewed from above) increases the value of
244-
`rot_y`, while turning it anticlockwise decreases it. A value of 0 means
245-
that the marker is perpendicular to the line of sight of the camera.
209+
rot_x
210+
: Rotation around the X-axis, in radians.
246211

247-
rot_z
248-
: Rotation of the marker about the z-axis.
212+
rot_y
213+
: Rotation around the Y-axis, in radians.
249214

250-
Turning a marker anticlockwise (as viewed from the camera) increases the
251-
value of `rot_z`, while turning it clockwise decreases it. A value of 0
252-
indicates that the marker is upright.
215+
dist
216+
: Distance, in metres.
253217

254-
<!---
255-
It would be nice to be able to include here what happens to the values:
256-
* what about if a marker is upside down but also leant forwards
257-
* if a marker is seen at a side angle, but is also leant forwards, does
258-
the value of rot_x measure its lean _towards the camera_ or from its
259-
_own vertical_?
260-
-->
218+
The camera is located at the origin, where the coordinates are (0, 0, 0).

0 commit comments

Comments
 (0)