AOMedia AV1 Codec
Source Frame Processing

Functions

void av1_tf_do_filtering_row (struct AV1_COMP *cpi, struct ThreadData *td, int mb_row)
 Does temporal filter for a given macroblock row.
 
void av1_temporal_filter (struct AV1_COMP *cpi, const int filter_frame_lookahead_idx, int gf_frame_index, FRAME_DIFF *frame_diff, YV12_BUFFER_CONFIG *output_frame)
 Performs temporal filtering if needed on a source frame. For example to create a filtered alternate reference frame (ARF)
 
int av1_check_show_filtered_frame (const YV12_BUFFER_CONFIG *frame, const FRAME_DIFF *frame_diff, int q_index, aom_bit_depth_t bit_depth)
 Check whether a filtered frame can be show directly.
 
static void tf_motion_search (AV1_COMP *cpi, MACROBLOCK *mb, const YV12_BUFFER_CONFIG *frame_to_filter, const YV12_BUFFER_CONFIG *ref_frame, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, MV *ref_mv, bool allow_me_for_sub_blks, MV *subblock_mvs, int *subblock_mses)
 Does motion search for blocks in temporal filtering. This is the first step for temporal filtering. More specifically, given a frame to be filtered and another frame as reference, this function searches the reference frame to find out the most similar block as that from the frame to be filtered. This found block will be further used for weighted averaging.
 
static void tf_build_predictor (const YV12_BUFFER_CONFIG *ref_frame, const MACROBLOCKD *mbd, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, const int num_planes, const struct scale_factors *scale, const MV *subblock_mvs, uint8_t *pred)
 Builds predictor for blocks in temporal filtering. This is the second step for temporal filtering, which is to construct predictions from all reference frames INCLUDING the frame to be filtered itself. These predictors are built based on the motion search results (motion vector is set as 0 for the frame to be filtered), and will be futher used for weighted averaging.
 
void av1_apply_temporal_filter_c (const YV12_BUFFER_CONFIG *frame_to_filter, const MACROBLOCKD *mbd, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, const int num_planes, const double *noise_levels, const MV *subblock_mvs, const int *subblock_mses, const int q_factor, const int filter_strength, int tf_wgt_calc_lvl, const uint8_t *pred, uint32_t *accum, uint16_t *count)
 Applies temporal filtering. NOTE that there are various optimised versions of this function called where the appropriate instruction set is supported.
 
static void tf_normalize_filtered_frame (const MACROBLOCKD *mbd, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, const int num_planes, const uint32_t *accum, const uint16_t *count, YV12_BUFFER_CONFIG *result_buffer)
 Normalizes the accumulated filtering result to produce the filtered frame.
 
static void tf_do_filtering (AV1_COMP *cpi)
 Does temporal filter for a given frame.
 
static void tf_setup_filtering_buffer (AV1_COMP *cpi, int filter_frame_lookahead_idx, int gf_frame_index)
 Setups the frame buffer for temporal filtering. This fuction determines how many frames will be used for temporal filtering and then groups them into a buffer. This function will also estimate the noise level of the to-filter frame.
 

Detailed Description

This module describes algorithms in AV1 assosciated with the pre-processing of source frames. See also Source Frame Processing

Function Documentation

◆ av1_tf_do_filtering_row()

◆ av1_temporal_filter()

void av1_temporal_filter ( struct AV1_COMP * cpi,
const int filter_frame_lookahead_idx,
int gf_frame_index,
FRAME_DIFF * frame_diff,
YV12_BUFFER_CONFIG * output_frame )

Performs temporal filtering if needed on a source frame. For example to create a filtered alternate reference frame (ARF)

In this function, the lookahead index is different from the 0-based real index. For example, if we want to filter the first frame in the pre-fetched buffer cpi->lookahead, the lookahead index will be -1 instead of 0. More concretely, 0 indicates the first LOOKAHEAD frame, which is the second frame in the pre-fetched buffer. Another example: if we want to filter the 17-th frame, which is an ARF, the lookahead index is 15 instead of 16. Futhermore, negative number is used for key frame in one-pass mode, where key frame is filtered with the frames before it instead of after it. For example, -15 means to filter the 17-th frame, which is a key frame in one-pass mode.

Parameters
[in]cpiTop level encoder instance structure
[in]filter_frame_lookahead_idxThe index of the to-filter frame in the lookahead buffer cpi->lookahead.
[in]gf_frame_indexIndex of GOP
[in,out]frame_diffstructure of sse and sum of the filtered frame.
[out]output_frameOuput filtered frame.

◆ av1_check_show_filtered_frame()

int av1_check_show_filtered_frame ( const YV12_BUFFER_CONFIG * frame,
const FRAME_DIFF * frame_diff,
int q_index,
aom_bit_depth_t bit_depth )

Check whether a filtered frame can be show directly.

This function will use the filtered frame's sse and current q index to make decision.

Parameters
[in]framefiltered frame's buffer
[in]frame_diffstructure of sse and sum of the filtered frame.
[in]q_indexq_index used for this frame
[in]bit_depthbit depth
Returns
return 1 if this frame can be shown directly, otherwise return 0

◆ tf_motion_search()

static void tf_motion_search ( AV1_COMP * cpi,
MACROBLOCK * mb,
const YV12_BUFFER_CONFIG * frame_to_filter,
const YV12_BUFFER_CONFIG * ref_frame,
const BLOCK_SIZE block_size,
const int mb_row,
const int mb_col,
MV * ref_mv,
bool allow_me_for_sub_blks,
MV * subblock_mvs,
int * subblock_mses )
static

Does motion search for blocks in temporal filtering. This is the first step for temporal filtering. More specifically, given a frame to be filtered and another frame as reference, this function searches the reference frame to find out the most similar block as that from the frame to be filtered. This found block will be further used for weighted averaging.

NOTE: Besides doing motion search for the entire block, this function will also do motion search for each 1/4 sub-block to get more precise predictions. Then, this function will determines whether to use 4 sub-blocks to replace the entire block. If we do need to split the entire block, 4 elements in subblock_mvs and subblock_mses refer to the searched motion vector and search error (MSE) w.r.t. each sub-block respectively. Otherwise, the 4 elements will be the same, all of which are assigned as the searched motion vector and search error (MSE) for the entire block.

Parameters
[in]cpiTop level encoder instance structure
[in]mbPointer to macroblock
[in]frame_to_filterPointer to the frame to be filtered
[in]ref_framePointer to the reference frame
[in]block_sizeBlock size used for motion search
[in]mb_rowRow index of the block in the frame
[in]mb_colColumn index of the block in the frame
[in]ref_mvReference motion vector, which is commonly inherited from the motion search result of previous frame.
[in]allow_me_for_sub_blksFlag to indicate whether motion search at 16x16 sub-block level is needed or not.
[out]subblock_mvsPointer to the motion vectors for 4 sub-blocks
[out]subblock_msesPointer to the search errors (MSE) for 4 sub-blocks
Remarks
Nothing will be returned. Results are saved in subblock_mvs and subblock_mses

References macroblockd::bd, AV1_COMP::common, FeatureFlags::cur_frame_force_integer_mv, macroblock::e_mbd, AV1Common::features, MotionVectorSearchParams::find_fractional_mv_step, AV1_PRIMARY::fn_ptr, get_q(), AV1Common::height, AV1_COMP::mv_search_params, SPEED_FEATURES::mv_sf, macroblock::plane, macroblockd::plane, AV1_COMP::ppi, PRUNE_MESH_SEARCH_LVL_1, AV1_COMP::sf, macroblock_plane::src, and AV1Common::width.

Referenced by av1_tf_do_filtering_row().

◆ tf_build_predictor()

static void tf_build_predictor ( const YV12_BUFFER_CONFIG * ref_frame,
const MACROBLOCKD * mbd,
const BLOCK_SIZE block_size,
const int mb_row,
const int mb_col,
const int num_planes,
const struct scale_factors * scale,
const MV * subblock_mvs,
uint8_t * pred )
static

Builds predictor for blocks in temporal filtering. This is the second step for temporal filtering, which is to construct predictions from all reference frames INCLUDING the frame to be filtered itself. These predictors are built based on the motion search results (motion vector is set as 0 for the frame to be filtered), and will be futher used for weighted averaging.

Parameters
[in]ref_framePointer to the reference frame (or the frame to be filtered)
[in]mbdPointer to the block for filtering. Besides containing the subsampling information of all planes, this field also gives the searched motion vector for the entire block, i.e., mbd->mi[0]->mv[0]. This vector should be 0 if the ref_frame itself is the frame to be filtered.
[in]block_sizeSize of the block
[in]mb_rowRow index of the block in the frame
[in]mb_colColumn index of the block in the frame
[in]num_planesNumber of planes in the frame
[in]scaleScaling factor
[in]subblock_mvsThe motion vectors for each sub-block (row-major order)
[out]predPointer to the predictor to be built
Remarks
Nothing returned, But the contents of pred will be modified

References macroblockd::bd, and macroblockd::plane.

Referenced by av1_tf_do_filtering_row().

◆ av1_apply_temporal_filter_c()

void av1_apply_temporal_filter_c ( const YV12_BUFFER_CONFIG * frame_to_filter,
const MACROBLOCKD * mbd,
const BLOCK_SIZE block_size,
const int mb_row,
const int mb_col,
const int num_planes,
const double * noise_levels,
const MV * subblock_mvs,
const int * subblock_mses,
const int q_factor,
const int filter_strength,
int tf_wgt_calc_lvl,
const uint8_t * pred,
uint32_t * accum,
uint16_t * count )

Applies temporal filtering. NOTE that there are various optimised versions of this function called where the appropriate instruction set is supported.

Parameters
[in]frame_to_filterPointer to the frame to be filtered, which is used as reference to compute squared difference from the predictor.
[in]mbdPointer to the block for filtering, ONLY used to get subsampling information for the planes
[in]block_sizeSize of the block
[in]mb_rowRow index of the block in the frame
[in]mb_colColumn index of the block in the frame
[in]num_planesNumber of planes in the frame
[in]noise_levelsEstimated noise levels for each plane in the frame (Y,U,V)
[in]subblock_mvsPointer to the motion vectors for 4 sub-blocks
[in]subblock_msesPointer to the search errors (MSE) for 4 sub-blocks
[in]q_factorQuantization factor. This is actually the q defined in libaom, converted from qindex
[in]filter_strengthFiltering strength. This value lies in range [0, 6] where 6 is the maximum strength.
[in]tf_wgt_calc_lvlControls the weight calculation method during temporal filtering
[out]predPointer to the well-built predictors
[out]accumPointer to the pixel-wise accumulator for filtering
[out]countPointer to the pixel-wise counter for filtering
Remarks
Nothing returned, But the contents of accum, pred and 'count' will be modified

< Y (Luminance) plane

< Y (Luminance) plane

< Y (Luminance) plane

< U (Chroma) plane

References AOM_CODEC_MEM_ERROR, AOM_PLANE_U, AOM_PLANE_Y, macroblockd::bd, macroblockd::error_info, and macroblockd::plane.

Referenced by av1_tf_do_filtering_row().

◆ tf_normalize_filtered_frame()

static void tf_normalize_filtered_frame ( const MACROBLOCKD * mbd,
const BLOCK_SIZE block_size,
const int mb_row,
const int mb_col,
const int num_planes,
const uint32_t * accum,
const uint16_t * count,
YV12_BUFFER_CONFIG * result_buffer )
static

Normalizes the accumulated filtering result to produce the filtered frame.

Parameters
[in]mbdPointer to the block for filtering, which is ONLY used to get subsampling information for all the planes
[in]block_sizeSize of the block
[in]mb_rowRow index of the block in the frame
[in]mb_colColumn index of the block in the frame
[in]num_planesNumber of planes in the frame
[in]accumPointer to the pre-computed accumulator
[in]countPointer to the pre-computed count
[out]result_bufferPointer to result buffer
Remarks
Nothing returned, but the content to which result_buffer pointer will be modified

References macroblockd::plane.

Referenced by av1_tf_do_filtering_row().

◆ tf_do_filtering()

static void tf_do_filtering ( AV1_COMP * cpi)
static

Does temporal filter for a given frame.

Parameters
[in]cpiTop level encoder instance structure
Remarks
Nothing will be returned, but the contents of td->diff will be modified.

References av1_tf_do_filtering_row(), AV1_COMP::common, TemporalFilterCtx::mb_rows, TemporalFilterCtx::sf, AV1_COMP::td, and AV1_COMP::tf_ctx.

◆ tf_setup_filtering_buffer()

static void tf_setup_filtering_buffer ( AV1_COMP * cpi,
int filter_frame_lookahead_idx,
int gf_frame_index )
static

Setups the frame buffer for temporal filtering. This fuction determines how many frames will be used for temporal filtering and then groups them into a buffer. This function will also estimate the noise level of the to-filter frame.

Parameters
[in]cpiTop level encoder instance structure
[in]filter_frame_lookahead_idxThe index of the to-filter frame in the lookahead buffer cpi->lookahead
[in]gf_frame_indexGOP index
Remarks
Nothing will be returned. But the fields frames, num_frames, filter_frame_idx and noise_levels will be updated in cpi->tf_ctx.

< Y (Luminance) plane

< Y (Luminance) plane

< Y (Luminance) plane

References HIGH_LEVEL_SPEED_FEATURES::adjust_num_frames_for_arf_filtering, AV1EncoderConfig::algo_cfg, AOM_PLANE_Y, AlgoCfg::arnr_max_frames, av1_gop_check_forward_keyframe(), av1_lookahead_depth(), av1_lookahead_peek(), AV1_COMP::common, AV1_COMP::compressor_stage, TemporalFilterCtx::filter_frame_idx, AV1_COMP::frame_info, TemporalFilterCtx::frames, RATE_CONTROL::frames_to_key, get_q(), AV1_PRIMARY::gf_group, SPEED_FEATURES::hl_sf, AV1_PRIMARY::lookahead, TemporalFilterCtx::noise_levels, TemporalFilterCtx::num_frames, AV1_COMP::oxcf, AV1_PRIMARY::p_rc, AV1_COMP::ppi, AV1_COMP::rc, AV1Common::seq_params, AV1_COMP::sf, AV1_COMP::td, AV1_COMP::tf_ctx, AV1_PRIMARY::twopass, and AV1_COMP::twopass_frame.