Historical timeline of video coding standards and formats

1. Who should read this?

The majority of video codecs in use today conform to one of the international standards for video coding. Therefore, if you are learning about video codecs or into codec development, it is important to know what a video coding standard is and how the existing standards have evolved. Even the latest video codecs such as H.265/HEVC and H.266/VVC have a basic structure similar to older codecs such as MPEG-2 part-2. This white paper begins by describing a video coding standard and the need for standardisation. The process of standardisation is briefly explained. The remainder of the paper provides a timeline of development of popular video coding standards and video coding formats.

2. What is a video coding standard?

A video coding standard is a document describing a bitstream structure and a decoding method for video compression. The standard does not define the encoder; rather it defines the output structure that an encoder should produce. Video coding standards usually define a toolkit or a set of tools for compression. Not all pieces of the toolkit need to be implemented to create a standard compliant bitstream. However, decoders conforming to the standard must implement some subset of the toolkit. The popular H.26x and MPEG-x series of standards are published by the ITU-T and ISO/IEC international standards bodies.

3. Why standardize?

Standardisation enables encoders and decoders from different manufacturers to work together across a range of applications. In order to provide the maximum flexibility and encourage innovation, the standards do not include all practical issues and design techniques of the decoding process. Standardisation allows for decoding of bit streams from previous and emerging standards. There is also a greater freedom for consumers to choose between manufacturers. For developers of standard-compliant video coding systems, the published standard is an essential point of reference as it defines the capabilities that a video codec must conform to, in order to successfully interwork with other systems.

4. The standardisation process

The main steps towards the finalisation of a standard are shown in Figure 1.

Figure 1 - The standardisation process

In the first phase, the requirements for a specific application or a field of applications are identified. The next phase involves the development of different algorithms by various laboratories, companies or contributors. The developed algorithms are compared and a set of basic techniques, which forms the core of the standard, is selected and refined in a joint effort during the collaboration phase. At the end of this phase a draft standard is issued, which has to be validated by compliance testing based on computer simulations or hardware tests and field trials. After successful validation and refinements, the final standard is published. A committee of video coding experts, typically a subcommittee of the Moving Picture Experts Group (MPEG) or the Video Coding Experts Group (VCEG), or a joint committee of both groups, is responsible for identifying the requirements for a new standard and managing the process of selecting and refining the technologies within the emerging standard.

5. The timeline

Two standardisation bodies, the international standards organisation (ISO) and the international telecommunications union (ITU), have developed a series of standards that have shaped the development of the media industry. Popular ISO coding standards include MPEG- 1, MPEG-2 and MPEG-4. ITU-T has published the H.26x line of coding standards including H.261, H.262, H.263 and H.263+. Other video coding formats such as Theora, VC-1, VPx and Daala have been developed over the years. The timeline of major standards and video formats over the last 25 years is shown in figure 2.

Figure 2 - Timeline for various video coding standards and formats

Many of the basic concepts of video coding such as transform coding, motion estimation and compensation and entropy coding were developed in the 1970s and 1980s. MPEG-1 was standardised on the early 1990s. MPEG-4 was developed in the late 1990s and soon after, H.263 was standardised. H.264/AVC was published in 2003 and Google released the VP8 video coding format in 2010. 2013 saw the publication of the H.265/High Efficiency Video Coding (HEVC) standard and Google’s VP9 format. An early version of the AV1 video coding format was released in 2016 and the Versatile Video Video Coding (H.266) standard was published in 2020.

The following sections cover these popular video coding standards and formats in detail.

6. The MPEG family of video coding standards

The ISO/IEC Motion Picture Experts Group (MPEG) publishes standards for media coding, storage and transport. These standards consist of different Parts. Each part covers a certain aspect of the whole specification. In this article, we discuss the parts related to video. Table 1 lists the key features of popular MPEG video coding standards.

Table 1 - MPEG family

7. The H26x family of video coding standards

The video coding experts group (VCEG), a working group of the ITU-T, published early standards in the H.26x family such as H.261, H.262 and H.263. The H.264/AVC or MPEG-4 Part-10 video coding standard was co-developed by the Joint Video Team (JVT), a collaboration between VCEG and MPEG. The H265/High Efficiency Video Coding (HEVC) standard was developed by the Joint Collaborative Team on Video Coding (JCT-VC) and the latest standard is H.266/Versatile Video Coding. Table 2 gives a high level overview of the key features of the H.26x standards.

Table 2 - H.26x series of standards

8. VC-1 standard

VC-1 is a video codec specification standardised by the Society of Motion Picture and Television Engineers (SMPTE) in 2006. It was implemented by Microsoft as Microsoft Windows Media Video (WMV) 9. Table 3 lists some key features of the VC-1 standard.

Table 3 - VC-1 coding standard

9. Other popular video coding formats

This section highlights some of the key features of other popular video coding formats such as VC-1, VP8, VP9, Theora and Daala. VC-1 is a video codec specification standardised by the Society of Motion Picture and Television Engineers (SMPTE) in 2006. It was implemented by Microsoft as Microsoft Windows Media Video (WMV). In May 2010, Google announced the start of a new open media project called WebM “with the goal of providing the web with a high-quality, open, royalty-free video codec that anyone can use, and that can inspire future innovators” [12]. At the core of the project were the VP8 and VP9 compression formats. In 2016, the first draft of the AV1 video coding format was published by the Alliance for Open Media (AOM).

Table 4 highlights some of the features of these video coding formats.

Table 4 - Other popular video coding formats

About the author

Vcodex is led by Professor Iain Richardson, an internationally known expert on the MPEG and H.264 video compression standards. Based in Delft, The Netherlands, he frequently travels to the US and Europe.

Iain Richardson is an internationally recognised expert on video compression and digital video communications. He is the author of four other books about video coding which include two widely-cited books on the H.264 Advanced Video Coding standard. For over thirty years, he has carried out research in the field of video compression and video communications, as a Professor at the Robert Gordon University in Aberdeen, Scotland and as an independent consultant with his own company, Vcodex. He advises companies on video compression technology and is sought after as an expert witness in litigation cases involving video coding.