1. Introduction

This document describes the methods of filtering reconstructed blocks in an H.264 video compression codec.

2. Description of reconstruction filter

 A filter can be applied to every decoded macroblock in order to reduce blocking distortion. The deblocking filter is applied after the inverse transform in the encoder (before reconstructing and storing the macroblock for future predictions) and in the decoder (before reconstructing and displaying the macroblock). The filter has two benefits: (1) block edges are smoothed, improving the appearance of decoded images (particularly at higher compression ratios) and (2) the filtered macroblock is used for motion-compensated prediction of further frames in the encoder, resulting in a smaller residual after prediction. (Note: intra-coded macroblocks are filtered, but intra prediction is carried out using unfiltered reconstructed macroblocks to form the prediction). Picture edges are not filtered.

Filtering is applied to vertical or horizontal edges of 4x4 blocks in a macroblock, in the following order:

1. Filter 4 vertical boundaries of the luma component (in order a,b,c,d in Figure 1)

2. Filter 4 horizontal boundaries of the luma component (in order e,f,g,h, Figure 1)

3. Filter 2 vertical boundaries of each chroma component (i,j)

4. Filter 2 horizontal boundaries of each chroma component (k,l)

Each filtering operation affects up to three pixels on either side of the boundary. Figure 2 shows 4 pixels on either side of a vertical or horizontal boundary in adjacent blocks p and q (p0,p1,p2,p3 and q0,q1,q2,q3). Depending on the current quantizer, the coding modes of neighbouring blocks and the gradient of image samples across the boundary, several outcomes are possible, ranging from (a) no pixels are filtered to (b) p0, p1, p2, q0, q1, q2 are filtered to produce output pixels P0, P1, P2, Q0, Q1 and Q2.

3. Boundary strength

The choice of filtering outcome depends on the boundary strength and on the gradient of image samples across the boundary. The boundary strength parameter Bs is chosen according to the following rules:

The filter is “stronger” at places where there is likely to be significant blocking distortion, such as the boundary of an intra coded macroblock or a boundary between blocks that contain coded coefficients.

4. Filter decision 

A group of samples from the set (p2,p1,p0,q0,q1,q2) is filtered only if:

(a)  Bs > 0 and

(b)  ) |p0-q0|, |p1-p0| and |q1-q0| are each less than a threshold α or β (α and β are defined in the standard [i]).

The thresholds α and β increase with the average quantizer parameter QP of the two blocks p and q. The purpose of the filter decision is to “switch off” the filter when there is    significant change (gradient) across the block boundary in the original image. The definition of a significant change depends on QP. When QP is small, anything other than a very small gradient across the boundary likely to be due to image features (rather than blocking effects) that should be preserved and so the thresholds α and β are low. When QP is larger, blocking distortion is likely to be more significant and α, β are higher so that more filtering takes place.

5. Filter implementation 

(a) Bs ∈ {1,2,3}:

A 4-tap linear filter is applied with inputs p1, p0, q0 and q1, producing filtered outputs P0 and Q0(0<Bs<4).

In addition, if |p2-p0| is less than threshold β, a 4-tap linear filter is applied with inputs p2, p1, p0 and q0, producing filtered output P1. If |q2-q0| is less than threshold β, a 4-tap linear filter is applied with inputs q2, q1, q0 and p0, producing filtered output Q1. (p1 and q1 are never filtered for chroma, only for luma data).

(b) Bs = 4:

If |p2-p0|<β and |p0-q0|<round(α/4):

P0 is produced by 5-tap filtering of p2, p1, p0, q0 and q1 P1 is produced by 4-tap filtering of p2, p1, p0 and q0

(Luma only) P2 is produced by 5-tap filtering of p3, p2, p1, p0 and q0.

else:

P0 is produced by 3-tap filtering of p1, p0 and q1.

If |q2-q0|<β and |p0-q0|<round(α/4):

Q0 is produced by 5-tap filtering of q2, q1, q0, p0 and p1

Q1 is produced by 4-tap filtering of q2, q1, q0 and p0

(Luma only) Q2 is produced by 5-tap filtering of q3, q2, q1, q0 and p0.

else:

Q0 is produced by 3-tap filtering of q1, q0 and p1.

6. Filtering example

A QCIF video clip is encoded using the AVC reference software with a fixed Quantization Parameter of 32. Figure 3 shows an original frame from the clip; Figure 4 shows the same frame after inter coding and reconstruction, with the loop filter disabled. With the loop filter enabled (Figure 5) the appearance is considerably better; there is still some distortion but most of the block edges have disappeared or faded. Note that sharp contrast boundaries tend to be preserved by the filter whilst block edges in smoother regions of the picture are smoothed.

Figure 6 shows a decoded frame with a higher QP of 36 and with the loop filter disabled (note the increased blocking artefacts) and Figure 7 shows the same frame with the loop filter enabled.

Further reading

Iain E. Richardson, “The H.264 Advanced Video Compression Standard”, John Wiley & Sons, 2010.

Iain E. Richardson, “Coding Video: A Practical Guide to HEVC and Beyond”, John Wiley & Sons, 2024.

About the author

Vcodex is led by Professor Iain Richardson, an internationally known expert on the MPEG and H.264 video compression standards. Based in Delft, The Netherlands, he frequently travels to the US and Europe.

Iain Richardson is an internationally recognised expert on video compression and digital video communications. He is the author of four other books about video coding which include two widely-cited books on the H.264 Advanced Video Coding standard. For over thirty years, he has carried out research in the field of video compression and video communications, as a Professor at the Robert Gordon University in Aberdeen, Scotland and as an independent consultant with his own company, Vcodex. He advises companies on video compression technology and is sought after as an expert witness in litigation cases involving video coding.