summaryrefslogtreecommitdiff
path: root/Documentation/userspace-api/media/v4l/crop.rst
blob: 3fe185e25ccff5f6c4d9d6a3a6e1637f43bc7c53 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later

.. _crop:

*****************************************************
Image Cropping, Insertion and Scaling -- the CROP API
*****************************************************

.. note::

   The CROP API is mostly superseded by the newer :ref:`SELECTION API
   <selection-api>`. The new API should be preferred in most cases,
   with the exception of pixel aspect ratio detection, which is
   implemented by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` and has no
   equivalent in the SELECTION API. See :ref:`selection-vs-crop` for a
   comparison of the two APIs.

Some video capture devices can sample a subsection of the picture and
shrink or enlarge it to an image of arbitrary size. We call these
abilities cropping and scaling. Some video output devices can scale an
image up or down and insert it at an arbitrary scan line and horizontal
offset into a video signal.

Applications can use the following API to select an area in the video
signal, query the default area and the hardware limits.

.. note::

   Despite their name, the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`,
   :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP
   <VIDIOC_G_CROP>` ioctls apply to input as well as output devices.

Scaling requires a source and a target. On a video capture or overlay
device the source is the video signal, and the cropping ioctls determine
the area actually sampled. The target are images read by the application
or overlaid onto the graphics screen. Their size (and position for an
overlay) is negotiated with the :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>`
and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctls.

On a video output device the source are the images passed in by the
application, and their size is again negotiated with the
:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
ioctls, or may be encoded in a compressed video stream. The target is
the video signal, and the cropping ioctls determine the area where the
images are inserted.

Source and target rectangles are defined even if the device does not
support scaling or the :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and
:ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>` ioctls. Their size (and position
where applicable) will be fixed in this case.

.. note::

   All capture and output devices that support the CROP or SELECTION
   API will also support the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`
   ioctl.

Cropping Structures
===================


.. _crop-scale:

.. kernel-figure:: crop.svg
    :alt:    crop.svg
    :align:  center

    Image Cropping, Insertion and Scaling

    The cropping, insertion and scaling process



For capture devices the coordinates of the top left corner, width and
height of the area which can be sampled is given by the ``bounds``
substructure of the struct :c:type:`v4l2_cropcap` returned
by the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` ioctl. To support a wide
range of hardware this specification does not define an origin or units.
However by convention drivers should horizontally count unscaled samples
relative to 0H (the leading edge of the horizontal sync pulse, see
:ref:`vbi-hsync`). Vertically ITU-R line numbers of the first field
(see ITU R-525 line numbering for :ref:`525 lines <vbi-525>` and for
:ref:`625 lines <vbi-625>`), multiplied by two if the driver
can capture both fields.

The top left corner, width and height of the source rectangle, that is
the area actually sampled, is given by struct
:c:type:`v4l2_crop` using the same coordinate system as
struct :c:type:`v4l2_cropcap`. Applications can use the
:ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>`
ioctls to get and set this rectangle. It must lie completely within the
capture boundaries and the driver may further adjust the requested size
and/or position according to hardware limitations.

Each capture device has a default source rectangle, given by the
``defrect`` substructure of struct
:c:type:`v4l2_cropcap`. The center of this rectangle
shall align with the center of the active picture area of the video
signal, and cover what the driver writer considers the complete picture.
Drivers shall reset the source rectangle to the default when the driver
is first loaded, but not later.

For output devices these structures and ioctls are used accordingly,
defining the *target* rectangle where the images will be inserted into
the video signal.


Scaling Adjustments
===================

Video hardware can have various cropping, insertion and scaling
limitations. It may only scale up or down, support only discrete scaling
factors, or have different scaling abilities in horizontal and vertical
direction. Also it may not support scaling at all. At the same time the
struct :c:type:`v4l2_crop` rectangle may have to be aligned,
and both the source and target rectangles may have arbitrary upper and
lower size limits. In particular the maximum ``width`` and ``height`` in
struct :c:type:`v4l2_crop` may be smaller than the struct
:c:type:`v4l2_cropcap`. ``bounds`` area. Therefore, as
usual, drivers are expected to adjust the requested parameters and
return the actual values selected.

Applications can change the source or the target rectangle first, as
they may prefer a particular image size or a certain area in the video
signal. If the driver has to adjust both to satisfy hardware
limitations, the last requested rectangle shall take priority, and the
driver should preferably adjust the opposite one. The
:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl however shall not change
the driver state and therefore only adjust the requested rectangle.

Suppose scaling on a video capture device is restricted to a factor 1:1
or 2:1 in either direction and the target image size must be a multiple
of 16 × 16 pixels. The source cropping rectangle is set to defaults,
which are also the upper limit in this example, of 640 × 400 pixels at
offset 0, 0. An application requests an image size of 300 × 225 pixels,
assuming video will be scaled down from the "full picture" accordingly.
The driver sets the image size to the closest possible values 304 × 224,
then chooses the cropping rectangle closest to the requested size, that
is 608 × 224 (224 × 2:1 would exceed the limit 400). The offset 0, 0 is
still valid, thus unmodified. Given the default cropping rectangle
reported by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` the application can
easily propose another offset to center the cropping rectangle.

Now the application may insist on covering an area using a picture
aspect ratio closer to the original request, so it asks for a cropping
rectangle of 608 × 456 pixels. The present scaling factors limit
cropping to 640 × 384, so the driver returns the cropping size 608 × 384
and adjusts the image size to closest possible 304 × 192.


Examples
========

Source and target rectangles shall remain unchanged across closing and
reopening a device, such that piping data into or out of a device will
work without special preparations. More advanced applications should
ensure the parameters are suitable before starting I/O.

.. note::

   On the next two examples, a video capture device is assumed;
   change ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` for other types of device.

Example: Resetting the cropping parameters
==========================================

.. code-block:: c

    struct v4l2_cropcap cropcap;
    struct v4l2_crop crop;

    memset (&cropcap, 0, sizeof (cropcap));
    cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

    if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
	perror ("VIDIOC_CROPCAP");
	exit (EXIT_FAILURE);
    }

    memset (&crop, 0, sizeof (crop));
    crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    crop.c = cropcap.defrect;

    /* Ignore if cropping is not supported (EINVAL). */

    if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
	&& errno != EINVAL) {
	perror ("VIDIOC_S_CROP");
	exit (EXIT_FAILURE);
    }


Example: Simple downscaling
===========================

.. code-block:: c

    struct v4l2_cropcap cropcap;
    struct v4l2_format format;

    reset_cropping_parameters ();

    /* Scale down to 1/4 size of full picture. */

    memset (&format, 0, sizeof (format)); /* defaults */

    format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

    format.fmt.pix.width = cropcap.defrect.width >> 1;
    format.fmt.pix.height = cropcap.defrect.height >> 1;
    format.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;

    if (-1 == ioctl (fd, VIDIOC_S_FMT, &format)) {
	perror ("VIDIOC_S_FORMAT");
	exit (EXIT_FAILURE);
    }

    /* We could check the actual image size now, the actual scaling factor
       or if the driver can scale at all. */

Example: Selecting an output area
=================================

.. note:: This example assumes an output device.

.. code-block:: c

    struct v4l2_cropcap cropcap;
    struct v4l2_crop crop;

    memset (&cropcap, 0, sizeof (cropcap));
    cropcap.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;

    if (-1 == ioctl (fd, VIDIOC_CROPCAP;, &cropcap)) {
	perror ("VIDIOC_CROPCAP");
	exit (EXIT_FAILURE);
    }

    memset (&crop, 0, sizeof (crop));

    crop.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
    crop.c = cropcap.defrect;

    /* Scale the width and height to 50 % of their original size
       and center the output. */

    crop.c.width /= 2;
    crop.c.height /= 2;
    crop.c.left += crop.c.width / 2;
    crop.c.top += crop.c.height / 2;

    /* Ignore if cropping is not supported (EINVAL). */

    if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
	&& errno != EINVAL) {
	perror ("VIDIOC_S_CROP");
	exit (EXIT_FAILURE);
    }

Example: Current scaling factor and pixel aspect
================================================

.. note:: This example assumes a video capture device.

.. code-block:: c

    struct v4l2_cropcap cropcap;
    struct v4l2_crop crop;
    struct v4l2_format format;
    double hscale, vscale;
    double aspect;
    int dwidth, dheight;

    memset (&cropcap, 0, sizeof (cropcap));
    cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

    if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
	perror ("VIDIOC_CROPCAP");
	exit (EXIT_FAILURE);
    }

    memset (&crop, 0, sizeof (crop));
    crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

    if (-1 == ioctl (fd, VIDIOC_G_CROP, &crop)) {
	if (errno != EINVAL) {
	    perror ("VIDIOC_G_CROP");
	    exit (EXIT_FAILURE);
	}

	/* Cropping not supported. */
	crop.c = cropcap.defrect;
    }

    memset (&format, 0, sizeof (format));
    format.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

    if (-1 == ioctl (fd, VIDIOC_G_FMT, &format)) {
	perror ("VIDIOC_G_FMT");
	exit (EXIT_FAILURE);
    }

    /* The scaling applied by the driver. */

    hscale = format.fmt.pix.width / (double) crop.c.width;
    vscale = format.fmt.pix.height / (double) crop.c.height;

    aspect = cropcap.pixelaspect.numerator /
	 (double) cropcap.pixelaspect.denominator;
    aspect = aspect * hscale / vscale;

    /* Devices following ITU-R BT.601 do not capture
       square pixels. For playback on a computer monitor
       we should scale the images to this size. */

    dwidth = format.fmt.pix.width / aspect;
    dheight = format.fmt.pix.height;