QVideoFrame Class
The QVideoFrame class represents a frame of video data. More...
| Header: | #include <QVideoFrame> |
| qmake: | QT += multimedia |
Public Types
| enum | FieldType { ProgressiveFrame, TopField, BottomField, InterlacedFrame } |
| enum | PixelFormat { Format_Invalid, Format_ARGB32, Format_ARGB32_Premultiplied, Format_RGB32, Format_RGB24, …, Format_User } |
Public Functions
| QVideoFrame(const QVideoFrame &other) | |
| QVideoFrame(const QImage &image) | |
| QVideoFrame(int bytes, const QSize &size, int bytesPerLine, PixelFormat format) | |
| QVideoFrame(QAbstractVideoBuffer *buffer, const QSize &size, PixelFormat format) | |
| QVideoFrame() | |
| QVideoFrame & | operator=(const QVideoFrame &other) |
| ~QVideoFrame() | |
| QVariantMap | availableMetaData() const |
| uchar * | bits() |
| uchar * | bits(int plane) |
| const uchar * | bits() const |
| const uchar * | bits(int plane) const |
| QAbstractVideoBuffer * | buffer() const |
| int | bytesPerLine() const |
| int | bytesPerLine(int plane) const |
| qint64 | endTime() const |
| FieldType | fieldType() const |
| QVariant | handle() const |
| QAbstractVideoBuffer::HandleType | handleType() const |
| int | height() const |
| QImage | image() const |
| bool | isMapped() const |
| bool | isReadable() const |
| bool | isValid() const |
| bool | isWritable() const |
| bool | map(QAbstractVideoBuffer::MapMode mode) |
| QAbstractVideoBuffer::MapMode | mapMode() const |
| int | mappedBytes() const |
| QVariant | metaData(const QString &key) const |
| PixelFormat | pixelFormat() const |
| int | planeCount() const |
| void | setEndTime(qint64 time) |
| void | setFieldType(FieldType field) |
| void | setMetaData(const QString &key, const QVariant &value) |
| void | setStartTime(qint64 time) |
| QSize | size() const |
| qint64 | startTime() const |
| void | unmap() |
| int | width() const |
| bool | operator!=(const QVideoFrame &other) const |
| bool | operator==(const QVideoFrame &other) const |
Static Public Members
| QImage::Format | imageFormatFromPixelFormat(PixelFormat format) |
| PixelFormat | pixelFormatFromImageFormat(QImage::Format format) |
Detailed Description
A QVideoFrame encapsulates the pixel data of a video frame, and information about the frame.
Video frames can come from several places - decoded media, a camera, or generated programmatically. The way pixels are described in these frames can vary greatly, and some pixel formats offer greater compression opportunities at the expense of ease of use.
The pixel contents of a video frame can be mapped to memory using the map() function. While mapped, the video data can accessed using the bits() function, which returns a pointer to a buffer. The total size of this buffer is given by the mappedBytes() function, and the size of each line is given by bytesPerLine(). The return value of the handle() function may also be used to access frame data using the internal buffer's native APIs (for example - an OpenGL texture handle).
A video frame can also have timestamp information associated with it. These timestamps can be used by an implementation of QAbstractVideoSurface to determine when to start and stop displaying the frame, but not all surfaces might respect this setting.
The video pixel data in a QVideoFrame is encapsulated in a QAbstractVideoBuffer. A QVideoFrame may be constructed from any buffer type by subclassing the QAbstractVideoBuffer class.
Note: Since video frames can be expensive to copy, QVideoFrame is explicitly shared, so any change made to a video frame will also apply to any copies.
Member Type Documentation
enum QVideoFrame::FieldType
Specifies the field an interlaced video frame belongs to.
| Constant | Value | Description |
|---|---|---|
QVideoFrame::ProgressiveFrame | 0 | The frame is not interlaced. |
QVideoFrame::TopField | 1 | The frame contains a top field. |
QVideoFrame::BottomField | 2 | The frame contains a bottom field. |
QVideoFrame::InterlacedFrame | 3 | The frame contains a merged top and bottom field. |
enum QVideoFrame::PixelFormat
Enumerates video data types.
| Constant | Value | Description |
|---|---|---|
QVideoFrame::Format_Invalid | 0 | The frame is invalid. |
QVideoFrame::Format_ARGB32 | 1 | The frame is stored using a 32-bit ARGB format (0xAARRGGBB). This is equivalent to QImage::Format_ARGB32. |
QVideoFrame::Format_ARGB32_Premultiplied | 2 | The frame stored using a premultiplied 32-bit ARGB format (0xAARRGGBB). This is equivalent to QImage::Format_ARGB32_Premultiplied. |
QVideoFrame::Format_RGB32 | 3 | The frame stored using a 32-bit RGB format (0xffRRGGBB). This is equivalent to QImage::Format_RGB32 |
QVideoFrame::Format_RGB24 | 4 | The frame is stored using a 24-bit RGB format (8-8-8). This is equivalent to QImage::Format_RGB888 |
QVideoFrame::Format_RGB565 | 5 | The frame is stored using a 16-bit RGB format (5-6-5). This is equivalent to QImage::Format_RGB16. |
QVideoFrame::Format_RGB555 | 6 | The frame is stored using a 16-bit RGB format (5-5-5). This is equivalent to QImage::Format_RGB555. |
QVideoFrame::Format_ARGB8565_Premultiplied | 7 | The frame is stored using a 24-bit premultiplied ARGB format (8-5-6-5). |
QVideoFrame::Format_BGRA32 | 8 | The frame is stored using a 32-bit BGRA format (0xBBGGRRAA). |
QVideoFrame::Format_BGRA32_Premultiplied | 9 | The frame is stored using a premultiplied 32bit BGRA format. |
QVideoFrame::Format_ABGR32 | 33 | The frame is stored using a 32-bit ABGR format (0xAABBGGRR). |
QVideoFrame::Format_BGR32 | 10 | The frame is stored using a 32-bit BGR format (0xBBGGRRff). |
QVideoFrame::Format_BGR24 | 11 | The frame is stored using a 24-bit BGR format (0xBBGGRR). |
QVideoFrame::Format_BGR565 | 12 | The frame is stored using a 16-bit BGR format (5-6-5). |
QVideoFrame::Format_BGR555 | 13 | The frame is stored using a 16-bit BGR format (5-5-5). |
QVideoFrame::Format_BGRA5658_Premultiplied | 14 | The frame is stored using a 24-bit premultiplied BGRA format (5-6-5-8). |
QVideoFrame::Format_AYUV444 | 15 | The frame is stored using a packed 32-bit AYUV format (0xAAYYUUVV). |
QVideoFrame::Format_AYUV444_Premultiplied | 16 | The frame is stored using a packed premultiplied 32-bit AYUV format (0xAAYYUUVV). |
QVideoFrame::Format_YUV444 | 17 | The frame is stored using a 24-bit packed YUV format (8-8-8). |
QVideoFrame::Format_YUV420P | 18 | The frame is stored using an 8-bit per component planar YUV format with the U and V planes horizontally and vertically sub-sampled, i.e. the height and width of the U and V planes are half that of the Y plane. |
QVideoFrame::Format_YUV422P | 34 | The frame is stored using an 8-bit per component planar YUV format with the U and V planes horizontally sub-sampled, i.e. the width of the U and V planes are half that of the Y plane, and height of U and V planes is the same as Y. |
QVideoFrame::Format_YV12 | 19 | The frame is stored using an 8-bit per component planar YVU format with the V and U planes horizontally and vertically sub-sampled, i.e. the height and width of the V and U planes are half that of the Y plane. |
QVideoFrame::Format_UYVY | 20 | The frame is stored using an 8-bit per component packed YUV format with the U and V planes horizontally sub-sampled (U-Y-V-Y), i.e. two horizontally adjacent pixels are stored as a 32-bit macropixel which has a Y value for each pixel and common U and V values. |
QVideoFrame::Format_YUYV | 21 | The frame is stored using an 8-bit per component packed YUV format with the U and V planes horizontally sub-sampled (Y-U-Y-V), i.e. two horizontally adjacent pixels are stored as a 32-bit macropixel which has a Y value for each pixel and common U and V values. |
QVideoFrame::Format_NV12 | 22 | The frame is stored using an 8-bit per component semi-planar YUV format with a Y plane (Y) followed by a horizontally and vertically sub-sampled, packed UV plane (U-V). |
QVideoFrame::Format_NV21 | 23 | The frame is stored using an 8-bit per component semi-planar YUV format with a Y plane (Y) followed by a horizontally and vertically sub-sampled, packed VU plane (V-U). |
QVideoFrame::Format_IMC1 | 24 | The frame is stored using an 8-bit per component planar YUV format with the U and V planes horizontally and vertically sub-sampled. This is similar to the Format_YUV420P type, except that the bytes per line of the U and V planes are padded out to the same stride as the Y plane. |
QVideoFrame::Format_IMC2 | 25 | The frame is stored using an 8-bit per component planar YUV format with the U and V planes horizontally and vertically sub-sampled. This is similar to the Format_YUV420P type, except that the lines of the U and V planes are interleaved, i.e. each line of U data is followed by a line of V data creating a single line of the same stride as the Y data. |
QVideoFrame::Format_IMC3 | 26 | The frame is stored using an 8-bit per component planar YVU format with the V and U planes horizontally and vertically sub-sampled. This is similar to the Format_YV12 type, except that the bytes per line of the V and U planes are padded out to the same stride as the Y plane. |
QVideoFrame::Format_IMC4 | 27 | The frame is stored using an 8-bit per component planar YVU format with the V and U planes horizontally and vertically sub-sampled. This is similar to the Format_YV12 type, except that the lines of the V and U planes are interleaved, i.e. each line of V data is followed by a line of U data creating a single line of the same stride as the Y data. |
QVideoFrame::Format_Y8 | 28 | The frame is stored using an 8-bit greyscale format. |
QVideoFrame::Format_Y16 | 29 | The frame is stored using a 16-bit linear greyscale format. Little endian. |
QVideoFrame::Format_Jpeg | 30 | The frame is stored in compressed Jpeg format. |
QVideoFrame::Format_CameraRaw | 31 | The frame is stored using a device specific camera raw format. |
QVideoFrame::Format_AdobeDng | 32 | The frame is stored using raw Adobe Digital Negative (DNG) format. |
QVideoFrame::Format_User | 1000 | Start value for user defined pixel formats. |
Member Function Documentation
QVideoFrame::QVideoFrame(const QVideoFrame &other)
Constructs a shallow copy of other. Since QVideoFrame is explicitly shared, these two instances will reflect the same frame.
QVideoFrame::QVideoFrame(const QImage &image)
Constructs a video frame from an image.
Note: This will construct an invalid video frame if there is no frame type equivalent to the image format.
See also pixelFormatFromImageFormat().
QVideoFrame::QVideoFrame(int bytes, const QSize &size, int bytesPerLine, PixelFormat format)
Constructs a video frame of the given pixel format and size in pixels.
The bytesPerLine (stride) is the length of each scan line in bytes, and bytes is the total number of bytes that must be allocated for the frame.
QVideoFrame::QVideoFrame(QAbstractVideoBuffer *buffer, const QSize &size, PixelFormat format)
Constructs a video frame from a buffer with the given pixel format and size in pixels.
Note: This doesn't increment the reference count of the video buffer.
QVideoFrame::QVideoFrame()
Constructs a null video frame.
QVideoFrame &QVideoFrame::operator=(const QVideoFrame &other)
Assigns the contents of other to this video frame. Since QVideoFrame is explicitly shared, these two instances will reflect the same frame.
QVideoFrame::~QVideoFrame()
Destroys a video frame.
QVariantMap QVideoFrame::availableMetaData() const
Returns any extra metadata associated with this frame.
uchar *QVideoFrame::bits()
Returns a pointer to the start of the frame data buffer.
This value is only valid while the frame data is mapped.
Changes made to data accessed via this pointer (when mapped with write access) are only guaranteed to have been persisted when unmap() is called and when the buffer has been mapped for writing.
See also map(), mappedBytes(), and bytesPerLine().
uchar *QVideoFrame::bits(int plane)
Returns a pointer to the start of the frame data buffer for a plane.
This value is only valid while the frame data is mapped.
Changes made to data accessed via this pointer (when mapped with write access) are only guaranteed to have been persisted when unmap() is called and when the buffer has been mapped for writing.
This function was introduced in Qt 5.4.
See also map(), mappedBytes(), bytesPerLine(), and planeCount().
const uchar *QVideoFrame::bits() const
Returns a pointer to the start of the frame data buffer.
This value is only valid while the frame data is mapped.
If the buffer was not mapped with read access, the contents of this buffer will initially be uninitialized.
See also map(), mappedBytes(), and bytesPerLine().
const uchar *QVideoFrame::bits(int plane) const
Returns a pointer to the start of the frame data buffer for a plane.
This value is only valid while the frame data is mapped.
If the buffer was not mapped with read access, the contents of this buffer will initially be uninitialized.
This function was introduced in Qt 5.4.
See also map(), mappedBytes(), bytesPerLine(), and planeCount().
QAbstractVideoBuffer *QVideoFrame::buffer() const
Returns underlying video buffer or null if there is none.
This function was introduced in Qt 5.13.
int QVideoFrame::bytesPerLine() const
Returns the number of bytes in a scan line.
Note: For planar formats this is the bytes per line of the first plane only. The bytes per line of subsequent planes should be calculated as per the frame pixel format.
This value is only valid while the frame data is mapped.
See also bits(), map(), and mappedBytes().
int QVideoFrame::bytesPerLine(int plane) const
Returns the number of bytes in a scan line of a plane.
This value is only valid while the frame data is mapped.
This function was introduced in Qt 5.4.
See also bits(), map(), mappedBytes(), and planeCount().
qint64 QVideoFrame::endTime() const
Returns the presentation time (in microseconds) when a frame should stop being displayed.
An invalid time is represented as -1.
See also setEndTime().
FieldType QVideoFrame::fieldType() const
Returns the field an interlaced video frame belongs to.
If the video is not interlaced this will return WholeFrame.
See also setFieldType().
QVariant QVideoFrame::handle() const
Returns a type specific handle to a video frame's buffer.
For an OpenGL texture this would be the texture ID.
See also QAbstractVideoBuffer::handle().
QAbstractVideoBuffer::HandleType QVideoFrame::handleType() const
Returns the type of a video frame's handle.
int QVideoFrame::height() const
Returns the height of a video frame.
QImage QVideoFrame::image() const
Based on the pixel format converts current video frame to image.
This function was introduced in Qt 5.15.
[static] QImage::Format QVideoFrame::imageFormatFromPixelFormat(PixelFormat format)
Returns an image format equivalent to a video frame pixel format. If there is no equivalent format QImage::Format_Invalid is returned instead.
Note: In general QImage does not handle YUV formats.
bool QVideoFrame::isMapped() const
Identifies if a video frame's contents are currently mapped to system memory.
This is a convenience function which checks that the MapMode of the frame is not equal to QAbstractVideoBuffer::NotMapped.
Returns true if the contents of the video frame are mapped to system memory, and false otherwise.
See also mapMode() and QAbstractVideoBuffer::MapMode.
bool QVideoFrame::isReadable() const
Identifies if the mapped contents of a video frame were read from the frame when it was mapped.
This is a convenience function which checks if the MapMode contains the QAbstractVideoBuffer::WriteOnly flag.
Returns true if the contents of the mapped memory were read from the video frame, and false otherwise.
See also mapMode() and QAbstractVideoBuffer::MapMode.
bool QVideoFrame::isValid() const
Identifies whether a video frame is valid.
An invalid frame has no video buffer associated with it.
Returns true if the frame is valid, and false if it is not.
bool QVideoFrame::isWritable() const
Identifies if the mapped contents of a video frame will be persisted when the frame is unmapped.
This is a convenience function which checks if the MapMode contains the QAbstractVideoBuffer::WriteOnly flag.
Returns true if the video frame will be updated when unmapped, and false otherwise.
Note: The result of altering the data of a frame that is mapped in read-only mode is undefined. Depending on the buffer implementation the changes may be persisted, or worse alter a shared buffer.
See also mapMode() and QAbstractVideoBuffer::MapMode.
bool QVideoFrame::map(QAbstractVideoBuffer::MapMode mode)
Maps the contents of a video frame to system (CPU addressable) memory.
In some cases the video frame data might be stored in video memory or otherwise inaccessible memory, so it is necessary to map a frame before accessing the pixel data. This may involve copying the contents around, so avoid mapping and unmapping unless required.
The map mode indicates whether the contents of the mapped memory should be read from and/or written to the frame. If the map mode includes the QAbstractVideoBuffer::ReadOnly flag the mapped memory will be populated with the content of the video frame when initially mapped. If the map mode includes the QAbstractVideoBuffer::WriteOnly flag the content of the possibly modified mapped memory will be written back to the frame when unmapped.
While mapped the contents of a video frame can be accessed directly through the pointer returned by the bits() function.
When access to the data is no longer needed, be sure to call the unmap() function to release the mapped memory and possibly update the video frame contents.
If the video frame has been mapped in read only mode, it is permissible to map it multiple times in read only mode (and unmap it a corresponding number of times). In all other cases it is necessary to unmap the frame first before mapping a second time.
Note: Writing to memory that is mapped as read-only is undefined, and may result in changes to shared data or crashes.
Returns true if the frame was mapped to memory in the given mode and false otherwise.
See also unmap(), mapMode(), and bits().
QAbstractVideoBuffer::MapMode QVideoFrame::mapMode() const
Returns the mode a video frame was mapped to system memory in.
See also map() and QAbstractVideoBuffer::MapMode.
int QVideoFrame::mappedBytes() const
Returns the number of bytes occupied by the mapped frame data.
This value is only valid while the frame data is mapped.
See also map().
QVariant QVideoFrame::metaData(const QString &key) const
Returns any metadata for this frame for the given key.
This might include frame specific information from a camera, or subtitles from a decoded video stream.
See the documentation for the relevant video frame producer for further information about available metadata.
See also setMetaData().
PixelFormat QVideoFrame::pixelFormat() const
Returns the color format of a video frame.
[static] PixelFormat QVideoFrame::pixelFormatFromImageFormat(QImage::Format format)
Returns a video pixel format equivalent to an image format. If there is no equivalent format QVideoFrame::InvalidType is returned instead.
Note: In general QImage does not handle YUV formats.
int QVideoFrame::planeCount() const
Returns the number of planes in the video frame.
This value is only valid while the frame data is mapped.
This function was introduced in Qt 5.4.
See also map().
void QVideoFrame::setEndTime(qint64 time)
Sets the presentation time (in microseconds) when a frame should stop being displayed.
An invalid time is represented as -1.
See also endTime().
void QVideoFrame::setFieldType(FieldType field)
Sets the field an interlaced video frame belongs to.
See also fieldType().
void QVideoFrame::setMetaData(const QString &key, const QVariant &value)
Sets the metadata for the given key to value.
If value is a null variant, any metadata for this key will be removed.
The producer of the video frame might use this to associate certain data with this frame, or for an intermediate processor to add information for a consumer of this frame.
See also metaData().
void QVideoFrame::setStartTime(qint64 time)
Sets the presentation time (in microseconds) when the frame should initially be displayed.
An invalid time is represented as -1.
See also startTime().
QSize QVideoFrame::size() const
Returns the dimensions of a video frame.
qint64 QVideoFrame::startTime() const
Returns the presentation time (in microseconds) when the frame should be displayed.
An invalid time is represented as -1.
See also setStartTime().
void QVideoFrame::unmap()
Releases the memory mapped by the map() function.
If the MapMode included the QAbstractVideoBuffer::WriteOnly flag this will persist the current content of the mapped memory to the video frame.
unmap() should not be called if map() function failed.
See also map().
int QVideoFrame::width() const
Returns the width of a video frame.
bool QVideoFrame::operator!=(const QVideoFrame &other) const
Returns true if this QVideoFrame and other do not reflect the same frame.
bool QVideoFrame::operator==(const QVideoFrame &other) const
Returns true if this QVideoFrame and other reflect the same frame.