Skip to content

Commit 094f391

Browse files
djrscallygregkh
djrscally
authored and
gregkh
committed
docs: usb: Add documentation for the UVC Gadget
The UVC Gadget function has become quite complex, but documentation for it is fairly sparse. Add some more detailed documentation to improve the situation. Signed-off-by: Daniel Scally <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Greg Kroah-Hartman <[email protected]>
1 parent abfc4fa commit 094f391

File tree

2 files changed

+353
-0
lines changed

2 files changed

+353
-0
lines changed

‎Documentation/usb/gadget_uvc.rst

Lines changed: 352 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,352 @@
1+
=======================
2+
Linux UVC Gadget Driver
3+
=======================
4+
5+
Overview
6+
--------
7+
The UVC Gadget driver is a driver for hardware on the *device* side of a USB
8+
connection. It is intended to run on a Linux system that has USB device-side
9+
hardware such as boards with an OTG port.
10+
11+
On the device system, once the driver is bound it appears as a V4L2 device with
12+
the output capability.
13+
14+
On the host side (once connected via USB cable), a device running the UVC Gadget
15+
driver *and controlled by an appropriate userspace program* should appear as a UVC
16+
specification compliant camera, and function appropriately with any program
17+
designed to handle them. The userspace program running on the device system can
18+
queue image buffers from a variety of sources to be transmitted via the USB
19+
connection. Typically this would mean forwarding the buffers from a camera sensor
20+
peripheral, but the source of the buffer is entirely dependent on the userspace
21+
companion program.
22+
23+
Configuring the device kernel
24+
-----------------------------
25+
The Kconfig options USB_CONFIGFS, USB_LIBCOMPOSITE, USB_CONFIGFS_F_UVC and
26+
USB_F_UVC must be selected to enable support for the UVC gadget.
27+
28+
Configuring the gadget through configfs
29+
---------------------------------------
30+
The UVC Gadget expects to be configured through configfs using the UVC function.
31+
This allows a significant degree of flexibility, as many of a UVC device's
32+
settings can be controlled this way.
33+
34+
Not all of the available attributes are described here. For a complete enumeration
35+
see Documentation/ABI/testing/configfs-usb-gadget-uvc
36+
37+
Assumptions
38+
~~~~~~~~~~~
39+
This section assumes that you have mounted configfs at `/sys/kernel/config` and
40+
created a gadget as `/sys/kernel/config/usb_gadget/g1`.
41+
42+
The UVC Function
43+
~~~~~~~~~~~~~~~~
44+
45+
The first step is to create the UVC function:
46+
47+
.. code-block:: bash
48+
49+
# These variables will be assumed throughout the rest of the document
50+
CONFIGFS="/sys/kernel/config"
51+
GADGET="$CONFIGFS/usb_gadget/g1"
52+
FUNCTION="$GADGET/functions/uvc.0"
53+
54+
mkdir -p $FUNCTION
55+
56+
Formats and Frames
57+
~~~~~~~~~~~~~~~~~~
58+
59+
You must configure the gadget by telling it which formats you support, as well
60+
as the frame sizes and frame intervals that are supported for each format. In
61+
the current implementation there is no way for the gadget to refuse to set a
62+
format that the host instructs it to set, so it is important that this step is
63+
completed *accurately* to ensure that the host never asks for a format that
64+
can't be provided.
65+
66+
Formats are created under the streaming/uncompressed and streaming/mjpeg configfs
67+
groups, with the framesizes created under the formats in the following
68+
structure:
69+
70+
::
71+
72+
uvc.0 +
73+
|
74+
+ streaming +
75+
|
76+
+ mjpeg +
77+
| |
78+
| + mjpeg +
79+
| |
80+
| + 720p
81+
| |
82+
| + 1080p
83+
|
84+
+ uncompressed +
85+
|
86+
+ yuyv +
87+
|
88+
+ 720p
89+
|
90+
+ 1080p
91+
92+
Each frame can then be configured with a width and height, plus the maximum
93+
buffer size required to store a single frame, and finally with the supported
94+
frame intervals for that format and framesize. Width and height are enumerated in
95+
units of pixels, frame interval in units of 100ns. To create the structure
96+
above with 2, 15 and 100 fps frameintervals for each framesize for example you
97+
might do:
98+
99+
.. code-block:: bash
100+
101+
create_frame() {
102+
# Example usage:
103+
# create_frame <width> <height> <group> <format name>
104+
105+
WIDTH=$1
106+
HEIGHT=$2
107+
FORMAT=$3
108+
NAME=$4
109+
110+
wdir=$FUNCTION/streaming/$FORMAT/$NAME/${HEIGHT}p
111+
112+
mkdir -p $wdir
113+
echo $WIDTH > $wdir/wWidth
114+
echo $HEIGHT > $wdir/wHeight
115+
echo $(( $WIDTH * $HEIGHT * 2 )) > $wdir/dwMaxVideoFrameBufferSize
116+
cat <<EOF > $wdir/dwFrameInterval
117+
666666
118+
100000
119+
5000000
120+
EOF
121+
}
122+
123+
create_frame 1280 720 mjpeg mjpeg
124+
create_frame 1920 1080 mjpeg mjpeg
125+
create_frame 1280 720 uncompressed yuyv
126+
create_frame 1920 1080 uncompressed yuyv
127+
128+
The only uncompressed format currently supported is YUYV, which is detailed at
129+
Documentation/userspace-api/media/v4l/pixfmt-packed.yuv.rst.
130+
131+
Color Matching Descriptors
132+
~~~~~~~~~~~~~~~~~~~~~~~~~~
133+
It's possible to specify some colometry information for each format you create.
134+
This step is optional, and default information will be included if this step is
135+
skipped; those default values follow those defined in the Color Matching Descriptor
136+
section of the UVC specification.
137+
138+
To create a Color Matching Descriptor, create a configfs item and set its three
139+
attributes to your desired settings and then link to it from the format you wish
140+
it to be associated with:
141+
142+
.. code-block:: bash
143+
144+
# Create a new Color Matching Descriptor
145+
146+
mkdir $FUNCTION/streaming/color_matching/yuyv
147+
pushd $FUNCTION/streaming/color_matching/yuyv
148+
149+
echo 1 > bColorPrimaries
150+
echo 1 > bTransferCharacteristics
151+
echo 4 > bMatrixCoefficients
152+
153+
popd
154+
155+
# Create a symlink to the Color Matching Descriptor from the format's config item
156+
ln -s $FUNCTION/streaming/color_matching/yuyv $FUNCTION/streaming/uncompressed/yuyv
157+
158+
For details about the valid values, consult the UVC specification. Note that a
159+
default color matching descriptor exists and is used by any format which does
160+
not have a link to a different Color Matching Descriptor. It's possible to
161+
change the attribute settings for the default descriptor, so bear in mind that if
162+
you do that you are altering the defaults for any format that does not link to
163+
a different one.
164+
165+
166+
Header linking
167+
~~~~~~~~~~~~~~
168+
169+
The UVC specification requires that Format and Frame descriptors be preceded by
170+
Headers detailing things such as the number and cumulative size of the different
171+
Format descriptors that follow. This and similar operations are acheived in
172+
configfs by linking between the configfs item representing the header and the
173+
config items representing those other descriptors, in this manner:
174+
175+
.. code-block:: bash
176+
177+
mkdir $FUNCTION/streaming/header/h
178+
179+
# This section links the format descriptors and their associated frames
180+
# to the header
181+
cd $FUNCTION/streaming/header/h
182+
ln -s ../../uncompressed/yuyv
183+
ln -s ../../mjpeg/mjpeg
184+
185+
# This section ensures that the header will be transmitted for each
186+
# speed's set of descriptors. If support for a particular speed is not
187+
# needed then it can be skipped here.
188+
cd ../../class/fs
189+
ln -s ../../header/h
190+
cd ../../class/hs
191+
ln -s ../../header/h
192+
cd ../../class/ss
193+
ln -s ../../header/h
194+
cd ../../../control
195+
mkdir header/h
196+
ln -s header/h class/fs
197+
ln -s header/h class/ss
198+
199+
200+
Extension Unit Support
201+
~~~~~~~~~~~~~~~~~~~~~~
202+
203+
A UVC Extension Unit (XU) basically provides a distinct unit to which control set
204+
and get requests can be addressed. The meaning of those control requests is
205+
entirely implementation dependent, but may be used to control settings outside
206+
of the UVC specification (for example enabling or disabling video effects). An
207+
XU can be inserted into the UVC unit chain or left free-hanging.
208+
209+
Configuring an extension unit involves creating an entry in the appropriate
210+
directory and setting its attributes appropriately, like so:
211+
212+
.. code-block:: bash
213+
214+
mkdir $FUNCTION/control/extensions/xu.0
215+
pushd $FUNCTION/control/extensions/xu.0
216+
217+
# Set the bUnitID of the Processing Unit as the source for this
218+
# Extension Unit
219+
echo 2 > baSourceID
220+
221+
# Set this XU as the source of the default output terminal. This inserts
222+
# the XU into the UVC chain between the PU and OT such that the final
223+
# chain is IT > PU > XU.0 > OT
224+
cat bUnitID > ../../terminal/output/default/baSourceID
225+
226+
# Flag some controls as being available for use. The bmControl field is
227+
# a bitmap with each bit denoting the availability of a particular
228+
# control. For example to flag the 0th, 2nd and 3rd controls available:
229+
echo 0x0d > bmControls
230+
231+
# Set the GUID; this is a vendor-specific code identifying the XU.
232+
echo -e -n "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10" > guidExtensionCode
233+
234+
popd
235+
236+
The bmControls attribute and the baSourceID attribute are multi-value attributes.
237+
This means that you may write multiple newline separated values to them. For
238+
example to flag the 1st, 2nd, 9th and 10th controls as being available you would
239+
need to write two values to bmControls, like so:
240+
241+
.. code-block:: bash
242+
243+
cat << EOF > bmControls
244+
0x03
245+
0x03
246+
EOF
247+
248+
The multi-value nature of the baSourceID attribute belies the fact that XUs can
249+
be multiple-input, though note that this currently has no significant effect.
250+
251+
The bControlSize attribute reflects the size of the bmControls attribute, and
252+
similarly bNrInPins reflects the size of the baSourceID attributes. Both
253+
attributes are automatically increased / decreased as you set bmControls and
254+
baSourceID. It is also possible to manually increase or decrease bControlSize
255+
which has the effect of truncating entries to the new size, or padding entries
256+
out with 0x00, for example:
257+
258+
::
259+
260+
$ cat bmControls
261+
0x03
262+
0x05
263+
264+
$ cat bControlSize
265+
2
266+
267+
$ echo 1 > bControlSize
268+
$ cat bmControls
269+
0x03
270+
271+
$ echo 2 > bControlSize
272+
$ cat bmControls
273+
0x03
274+
0x00
275+
276+
bNrInPins and baSourceID function in the same way.
277+
278+
Custom Strings Support
279+
~~~~~~~~~~~~~~~~~~~~~~
280+
281+
String descriptors that provide a textual description for various parts of a
282+
USB device can be defined in the usual place within USB configfs, and may then
283+
be linked to from the UVC function root or from Extension Unit directories to
284+
assign those strings as descriptors:
285+
286+
.. code-block:: bash
287+
288+
# Create a string descriptor in us-EN and link to it from the function
289+
# root. The name of the link is significant here, as it declares this
290+
# descriptor to be intended for the Interface Association Descriptor.
291+
# Other significant link names at function root are vs0_desc and vs1_desc
292+
# For the VideoStreaming Interface 0/1 Descriptors.
293+
294+
mkdir -p $GADGET/strings/0x409/iad_desc
295+
echo -n "Interface Associaton Descriptor" > $GADGET/strings/0x409/iad_desc/s
296+
ln -s $GADGET/strings/0x409/iad_desc $FUNCTION/iad_desc
297+
298+
# Because the link to a String Descriptor from an Extension Unit clearly
299+
# associates the two, the name of this link is not significant and may
300+
# be set freely.
301+
302+
mkdir -p $GADGET/strings/0x409/xu.0
303+
echo -n "A Very Useful Extension Unit" > $GADGET/strings/0x409/xu.0/s
304+
ln -s $GADGET/strings/0x409/xu.0 $FUNCTION/control/extensions/xu.0
305+
306+
The interrupt endpoint
307+
~~~~~~~~~~~~~~~~~~~~~~
308+
309+
The VideoControl interface has an optional interrupt endpoint which is by default
310+
disabled. This is intended to support delayed response control set requests for
311+
UVC (which should respond through the interrupt endpoint rather than tying up
312+
endpoint 0). At present support for sending data through this endpoint is missing
313+
and so it is left disabled to avoid confusion. If you wish to enable it you can
314+
do so through the configfs attribute:
315+
316+
.. code-block:: bash
317+
318+
echo 1 > $FUNCTION/control/enable_interrupt_ep
319+
320+
Bandwidth configuration
321+
~~~~~~~~~~~~~~~~~~~~~~~
322+
323+
There are three attributes which control the bandwidth of the USB connection.
324+
These live in the function root and can be set within limits:
325+
326+
.. code-block:: bash
327+
328+
# streaming_interval sets bInterval. Values range from 1..255
329+
echo 1 > $FUNCTION/streaming_interval
330+
331+
# streaming_maxpacket sets wMaxPacketSize. Valid values are 1024/2048/3072
332+
echo 3072 > $FUNCTION/streaming_maxpacket
333+
334+
# streaming_maxburst sets bMaxBurst. Valid values are 1..15
335+
echo 1 > $FUNCTION/streaming_maxburst
336+
337+
338+
The values passed here will be clamped to valid values according to the UVC
339+
specification (which depend on the speed of the USB connection). To understand
340+
how the settings influence bandwidth you should consult the UVC specifications,
341+
but a rule of thumb is that increasing the streaming_maxpacket setting will
342+
improve bandwidth (and thus the maximum possible framerate), whilst the same is
343+
true for streaming_maxburst provided the USB connection is running at SuperSpeed.
344+
Increasing streaming_interval will reduce bandwidth and framerate.
345+
346+
The userspace application
347+
-------------------------
348+
By itself, the UVC Gadget driver cannot do anything particularly interesting. It
349+
must be paired with a userspace program that responds to UVC control requests and
350+
fills buffers to be queued to the V4L2 device that the driver creates. How those
351+
things are achieved is implementation dependent and beyond the scope of this
352+
document, but a reference application can be found at https://gitlab.freedesktop.org/camera/uvc-gadget

‎Documentation/usb/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ USB support
1616
gadget_multi
1717
gadget_printer
1818
gadget_serial
19+
gadget_uvc
1920
gadget-testing
2021
iuu_phoenix
2122
mass-storage

0 commit comments

Comments
 (0)