Source Frame Processing

Functions
void	av1_tf_do_filtering_row (struct AV1_COMP *cpi, struct ThreadData *td, int mb_row)
	Does temporal filter for a given macroblock row.
void	av1_temporal_filter (struct AV1_COMP *cpi, const int filter_frame_lookahead_idx, int gf_frame_index, FRAME_DIFF *frame_diff, YV12_BUFFER_CONFIG *output_frame)
	Performs temporal filtering if needed on a source frame. For example to create a filtered alternate reference frame (ARF)
int	av1_check_show_filtered_frame (const YV12_BUFFER_CONFIG *frame, const FRAME_DIFF *frame_diff, int q_index, aom_bit_depth_t bit_depth)
	Check whether a filtered frame can be show directly.
static void	tf_motion_search (AV1_COMP *cpi, MACROBLOCK *mb, const YV12_BUFFER_CONFIG *frame_to_filter, const YV12_BUFFER_CONFIG *ref_frame, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, MV *ref_mv, bool allow_me_for_sub_blks, MV *subblock_mvs, int *subblock_mses)
	Does motion search for blocks in temporal filtering. This is the first step for temporal filtering. More specifically, given a frame to be filtered and another frame as reference, this function searches the reference frame to find out the most similar block as that from the frame to be filtered. This found block will be further used for weighted averaging.
static void	tf_build_predictor (const YV12_BUFFER_CONFIG *ref_frame, const MACROBLOCKD *mbd, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, const int num_planes, const struct scale_factors *scale, const MV *subblock_mvs, uint8_t *pred)
	Builds predictor for blocks in temporal filtering. This is the second step for temporal filtering, which is to construct predictions from all reference frames INCLUDING the frame to be filtered itself. These predictors are built based on the motion search results (motion vector is set as 0 for the frame to be filtered), and will be futher used for weighted averaging.
void	av1_apply_temporal_filter_c (const YV12_BUFFER_CONFIG *frame_to_filter, const MACROBLOCKD *mbd, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, const int num_planes, const double *noise_levels, const MV *subblock_mvs, const int *subblock_mses, const int q_factor, const int filter_strength, int tf_wgt_calc_lvl, const uint8_t *pred, uint32_t *accum, uint16_t *count)
	Applies temporal filtering. NOTE that there are various optimised versions of this function called where the appropriate instruction set is supported.
static void	tf_normalize_filtered_frame (const MACROBLOCKD *mbd, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, const int num_planes, const uint32_t *accum, const uint16_t *count, YV12_BUFFER_CONFIG *result_buffer)
	Normalizes the accumulated filtering result to produce the filtered frame.
static void	tf_do_filtering (AV1_COMP *cpi)
	Does temporal filter for a given frame.
static void	tf_setup_filtering_buffer (AV1_COMP *cpi, int filter_frame_lookahead_idx, int gf_frame_index)
	Setups the frame buffer for temporal filtering. This fuction determines how many frames will be used for temporal filtering and then groups them into a buffer. This function will also estimate the noise level of the to-filter frame.

Detailed Description

This module describes algorithms in AV1 assosciated with the pre-processing of source frames. See also Source Frame Processing

Function Documentation

av1_tf_do_filtering_row()

void av1_tf_do_filtering_row	(	struct AV1_COMP *	cpi,
		struct ThreadData *	td,
		int	mb_row )

Does temporal filter for a given macroblock row.

Parameters

[in]	cpi	Top level encoder instance structure
[in]	td	Pointer to thread data
[in]	mb_row	Macroblock row to be filtered filtering

Remarks

Nothing will be returned, but the contents of td->diff will be modified.

References AV1EncoderConfig::algo_cfg, HIGH_LEVEL_SPEED_FEATURES::allow_sub_blk_me_in_tf, AlgoCfg::arnr_strength, av1_apply_temporal_filter_c(), AV1_COMP::common, TemporalFilterCtx::compute_frame_diff, macroblock::e_mbd, TemporalFilterCtx::filter_frame_idx, AV1_PRIMARY::fn_ptr, TemporalFilterCtx::frames, SPEED_FEATURES::hl_sf, TemporalFilterCtx::mb_cols, AV1Common::mi_params, macroblock::mv_limits, TemporalFilterCtx::noise_levels, TemporalFilterCtx::num_frames, TemporalFilterCtx::num_pels, TemporalFilterCtx::output_frame, AV1_COMP::oxcf, macroblockd::plane, AV1_COMP::ppi, TemporalFilterCtx::q_factor, AV1_COMP::sf, TemporalFilterCtx::sf, tf_build_predictor(), AV1_COMP::tf_ctx, tf_motion_search(), tf_normalize_filtered_frame(), and HIGH_LEVEL_SPEED_FEATURES::weight_calc_level_in_tf.

Referenced by tf_do_filtering().

av1_temporal_filter()

void av1_temporal_filter	(	struct AV1_COMP *	cpi,
		const int	filter_frame_lookahead_idx,
		int	gf_frame_index,
		FRAME_DIFF *	frame_diff,
		YV12_BUFFER_CONFIG *	output_frame )

Performs temporal filtering if needed on a source frame. For example to create a filtered alternate reference frame (ARF)

In this function, the lookahead index is different from the 0-based real index. For example, if we want to filter the first frame in the pre-fetched buffer cpi->lookahead, the lookahead index will be -1 instead of 0. More concretely, 0 indicates the first LOOKAHEAD frame, which is the second frame in the pre-fetched buffer. Another example: if we want to filter the 17-th frame, which is an ARF, the lookahead index is 15 instead of 16. Futhermore, negative number is used for key frame in one-pass mode, where key frame is filtered with the frames before it instead of after it. For example, -15 means to filter the 17-th frame, which is a key frame in one-pass mode.

Parameters

[in]	cpi	Top level encoder instance structure
[in]	filter_frame_lookahead_idx	The index of the to-filter frame in the lookahead buffer cpi->lookahead.
[in]	gf_frame_index	Index of GOP
[in,out]	frame_diff	structure of sse and sum of the filtered frame.
[out]	output_frame	Ouput filtered frame.

av1_check_show_filtered_frame()

int av1_check_show_filtered_frame	(	const YV12_BUFFER_CONFIG *	frame,
		const FRAME_DIFF *	frame_diff,
		int	q_index,
		aom_bit_depth_t	bit_depth )

Check whether a filtered frame can be show directly.

This function will use the filtered frame's sse and current q index to make decision.

Parameters

[in]	frame	filtered frame's buffer
[in]	frame_diff	structure of sse and sum of the filtered frame.
[in]	q_index	q_index used for this frame
[in]	bit_depth	bit depth

Returns

return 1 if this frame can be shown directly, otherwise return 0

tf_motion_search()

static void tf_motion_search ( AV1_COMP * cpi, MACROBLOCK * mb, const YV12_BUFFER_CONFIG * frame_to_filter, const YV12_BUFFER_CONFIG * ref_frame, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, MV * ref_mv, bool allow_me_for_sub_blks, MV * subblock_mvs, int * subblock_mses )

static

Does motion search for blocks in temporal filtering. This is the first step for temporal filtering. More specifically, given a frame to be filtered and another frame as reference, this function searches the reference frame to find out the most similar block as that from the frame to be filtered. This found block will be further used for weighted averaging.

NOTE: Besides doing motion search for the entire block, this function will also do motion search for each 1/4 sub-block to get more precise predictions. Then, this function will determines whether to use 4 sub-blocks to replace the entire block. If we do need to split the entire block, 4 elements in subblock_mvs and subblock_mses refer to the searched motion vector and search error (MSE) w.r.t. each sub-block respectively. Otherwise, the 4 elements will be the same, all of which are assigned as the searched motion vector and search error (MSE) for the entire block.

Parameters

[in]	cpi	Top level encoder instance structure
[in]	mb	Pointer to macroblock
[in]	frame_to_filter	Pointer to the frame to be filtered
[in]	ref_frame	Pointer to the reference frame
[in]	block_size	Block size used for motion search
[in]	mb_row	Row index of the block in the frame
[in]	mb_col	Column index of the block in the frame
[in]	ref_mv	Reference motion vector, which is commonly inherited from the motion search result of previous frame.
[in]	allow_me_for_sub_blks	Flag to indicate whether motion search at 16x16 sub-block level is needed or not.
[out]	subblock_mvs	Pointer to the motion vectors for 4 sub-blocks
[out]	subblock_mses	Pointer to the search errors (MSE) for 4 sub-blocks

Remarks

Nothing will be returned. Results are saved in subblock_mvs and subblock_mses

References macroblockd::bd, AV1_COMP::common, FeatureFlags::cur_frame_force_integer_mv, macroblock::e_mbd, AV1Common::features, MotionVectorSearchParams::find_fractional_mv_step, AV1_PRIMARY::fn_ptr, get_q(), AV1Common::height, AV1_COMP::mv_search_params, SPEED_FEATURES::mv_sf, macroblock::plane, macroblockd::plane, AV1_COMP::ppi, PRUNE_MESH_SEARCH_LVL_1, AV1_COMP::sf, macroblock_plane::src, and AV1Common::width.

Referenced by av1_tf_do_filtering_row().

tf_build_predictor()

static void tf_build_predictor ( const YV12_BUFFER_CONFIG * ref_frame, const MACROBLOCKD * mbd, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, const int num_planes, const struct scale_factors * scale, const MV * subblock_mvs, uint8_t * pred )

static

Builds predictor for blocks in temporal filtering. This is the second step for temporal filtering, which is to construct predictions from all reference frames INCLUDING the frame to be filtered itself. These predictors are built based on the motion search results (motion vector is set as 0 for the frame to be filtered), and will be futher used for weighted averaging.

Parameters

[in]	ref_frame	Pointer to the reference frame (or the frame to be filtered)
[in]	mbd	Pointer to the block for filtering. Besides containing the subsampling information of all planes, this field also gives the searched motion vector for the entire block, i.e., mbd->mi[0]->mv[0]. This vector should be 0 if the ref_frame itself is the frame to be filtered.
[in]	block_size	Size of the block
[in]	mb_row	Row index of the block in the frame
[in]	mb_col	Column index of the block in the frame
[in]	num_planes	Number of planes in the frame
[in]	scale	Scaling factor
[in]	subblock_mvs	The motion vectors for each sub-block (row-major order)
[out]	pred	Pointer to the predictor to be built

Remarks

Nothing returned, But the contents of pred will be modified

References macroblockd::bd, and macroblockd::plane.

Referenced by av1_tf_do_filtering_row().

av1_apply_temporal_filter_c()

void av1_apply_temporal_filter_c	(	const YV12_BUFFER_CONFIG *	frame_to_filter,
		const MACROBLOCKD *	mbd,
		const BLOCK_SIZE	block_size,
		const int	mb_row,
		const int	mb_col,
		const int	num_planes,
		const double *	noise_levels,
		const MV *	subblock_mvs,
		const int *	subblock_mses,
		const int	q_factor,
		const int	filter_strength,
		int	tf_wgt_calc_lvl,
		const uint8_t *	pred,
		uint32_t *	accum,
		uint16_t *	count )

Applies temporal filtering. NOTE that there are various optimised versions of this function called where the appropriate instruction set is supported.

Parameters

[in]	frame_to_filter	Pointer to the frame to be filtered, which is used as reference to compute squared difference from the predictor.
[in]	mbd	Pointer to the block for filtering, ONLY used to get subsampling information for the planes
[in]	block_size	Size of the block
[in]	mb_row	Row index of the block in the frame
[in]	mb_col	Column index of the block in the frame
[in]	num_planes	Number of planes in the frame
[in]	noise_levels	Estimated noise levels for each plane in the frame (Y,U,V)
[in]	subblock_mvs	Pointer to the motion vectors for 4 sub-blocks
[in]	subblock_mses	Pointer to the search errors (MSE) for 4 sub-blocks
[in]	q_factor	Quantization factor. This is actually the q defined in libaom, converted from qindex
[in]	filter_strength	Filtering strength. This value lies in range [0, 6] where 6 is the maximum strength.
[in]	tf_wgt_calc_lvl	Controls the weight calculation method during temporal filtering
[out]	pred	Pointer to the well-built predictors
[out]	accum	Pointer to the pixel-wise accumulator for filtering
[out]	count	Pointer to the pixel-wise counter for filtering

Remarks

Nothing returned, But the contents of accum, pred and 'count' will be modified

< Y (Luminance) plane

< Y (Luminance) plane

< Y (Luminance) plane

< U (Chroma) plane

References AOM_CODEC_MEM_ERROR, AOM_PLANE_U, AOM_PLANE_Y, macroblockd::bd, macroblockd::error_info, and macroblockd::plane.

Referenced by av1_tf_do_filtering_row().

tf_normalize_filtered_frame()

static void tf_normalize_filtered_frame ( const MACROBLOCKD * mbd, const BLOCK_SIZE block_size, const int mb_row, const int mb_col, const int num_planes, const uint32_t * accum, const uint16_t * count, YV12_BUFFER_CONFIG * result_buffer )

static

Normalizes the accumulated filtering result to produce the filtered frame.

Parameters

[in]	mbd	Pointer to the block for filtering, which is ONLY used to get subsampling information for all the planes
[in]	block_size	Size of the block
[in]	mb_row	Row index of the block in the frame
[in]	mb_col	Column index of the block in the frame
[in]	num_planes	Number of planes in the frame
[in]	accum	Pointer to the pre-computed accumulator
[in]	count	Pointer to the pre-computed count
[out]	result_buffer	Pointer to result buffer

Remarks

Nothing returned, but the content to which result_buffer pointer will be modified

References macroblockd::plane.

Referenced by av1_tf_do_filtering_row().

tf_do_filtering()

static void tf_do_filtering ( AV1_COMP * cpi )

static

Does temporal filter for a given frame.

Parameters

[in]

cpi

Top level encoder instance structure

Remarks

Nothing will be returned, but the contents of td->diff will be modified.

References av1_tf_do_filtering_row(), AV1_COMP::common, TemporalFilterCtx::mb_rows, TemporalFilterCtx::sf, AV1_COMP::td, and AV1_COMP::tf_ctx.

tf_setup_filtering_buffer()

static void tf_setup_filtering_buffer ( AV1_COMP * cpi, int filter_frame_lookahead_idx, int gf_frame_index )

static

Setups the frame buffer for temporal filtering. This fuction determines how many frames will be used for temporal filtering and then groups them into a buffer. This function will also estimate the noise level of the to-filter frame.

Parameters

[in]	cpi	Top level encoder instance structure
[in]	filter_frame_lookahead_idx	The index of the to-filter frame in the lookahead buffer cpi->lookahead
[in]	gf_frame_index	GOP index

Remarks

Nothing will be returned. But the fields frames, num_frames, filter_frame_idx and noise_levels will be updated in cpi->tf_ctx.

< Y (Luminance) plane

< Y (Luminance) plane

< Y (Luminance) plane

References HIGH_LEVEL_SPEED_FEATURES::adjust_num_frames_for_arf_filtering, AV1EncoderConfig::algo_cfg, AOM_PLANE_Y, AlgoCfg::arnr_max_frames, av1_gop_check_forward_keyframe(), av1_lookahead_depth(), av1_lookahead_peek(), AV1_COMP::common, AV1_COMP::compressor_stage, TemporalFilterCtx::filter_frame_idx, AV1_COMP::frame_info, TemporalFilterCtx::frames, RATE_CONTROL::frames_to_key, get_q(), AV1_PRIMARY::gf_group, SPEED_FEATURES::hl_sf, AV1_PRIMARY::lookahead, TemporalFilterCtx::noise_levels, TemporalFilterCtx::num_frames, AV1_COMP::oxcf, AV1_PRIMARY::p_rc, AV1_COMP::ppi, AV1_COMP::rc, AV1Common::seq_params, AV1_COMP::sf, AV1_COMP::td, AV1_COMP::tf_ctx, AV1_PRIMARY::twopass, and AV1_COMP::twopass_frame.