Transcript
Fraunhofer Image Processing Heinrich Hertz Institute
HEVC driving Ultra HD Ralf Schäfer
Fraunhofer Heinrich Hertz Institute, Einsteinufer 37, 10587 Berlin
www.hhi.fraunhofer.de
Image Processing
Motivation
UHDTV has a 10 x higher raw data rate than HDTV, but we want to use the same transmission channels (satellite, cable, Internet). -> A new compression system is required! pixels lines frame interlace chroma / line rate factor resolution
bit depth
raw data rate [Mbit/s]
HD
1920
1080
60
2:1
4:2:0
8
746
UHD-1
3840
2160
60
1:1
4:2:0
10
7465
©
Ralf Schäfer
2
Image Processing
The Development of Video Coding PSNR (dB) HEVC 40
(H.264/AVC, 2003)
(MPEG-2 1994)
(H.263, 1996) + (MPEG-4, 1998)
Target
38 Foreman 10 Hz, QCIF 100 frames
Bit-rate Reduction:75% 50% Bit-rate Reduction:
36 35 34 32
(H.261, 1991)
30 (JPEG, 1990)
28
bit rate (kbit/s) 0 ©
100
200 Ralf Schäfer
300
High Efficiency Video Coding (HEVC)
Image Processing
New “Joint Collaborative Team on Video Coding” (JCT-VC) of ISO/IEC MPEG and ITU-T VCEG April 2010: Call for Proposals evaluated by JCT-VC
27 proposals received
Significant video quality improvements relative to H.264/AVC High Profile for high and low delay
JCT-VC decided to create Test Model under Consideration (TMuC)
October 2010: Creation of first Test Model January 2013: Finalization of standard Document archives are publicly accessible
©
http://phenix.it-sudparis.eu/jct
http://ftp3.itu.ch/av-arch/jctvc-site
http://www.itu.int/ITU-T/studygroups/com16/jct-vc/index.html Ralf Schäfer
4
Basic Technology Architecture
Image Processing
HEVC is conceptually similar to H.264 (and prior standards)
Block-based Variable block sizes Block motion compensation Fractional-pel motion vectors Spatial intra prediction Spatial transform of residual difference Integer-based transform CABAC entropy coding In-loop filtering
Lots of variations at the individual “tool” level
©
Ralf Schäfer
5
Image Processing
Hybrid Video Coding Input Video Signal
Coder Control Transform/ Scal./Quant.
Split into Macroblocks 16x16 pixels
Control Data
Decoder
Quant. Transf. coeffs Scaling & Inv. Transform Entropy Coding
Intra-frame Prediction
Intra/Inter
MotionCompensation
De-blocking Filter
Output Video Signal Motion Data
Motion Estimation ©
Ralf Schäfer
6
Image Processing
Generalized Partitioning of Pictures
©
Picture is divided into Coding Tree Blocks of fixed size (e.g. 64x64) Root of the Coding Quadtree with Coding Block leaves Coding Block can be split • once into Prediction Block partitions • recursively into Transform Blocks using the Residual Quadtree (RQT) Maximum block size is signalled (e.g. 64x64) Minimum block size is signalled (e.g. 4x4) Rectangular shapes Ralf Schäfer
0
1
2
3
4
5
6
8
9
10
11
7
12
13
15
16
17
18
14
7
Profiles & Levels of HEVC
Image Processing
Version 1 of HEVC: January 2013 • Main Profile 4:2:0 chroma format, 8-bit sample bit depth • Main 10 Profile 4:2:0 chroma format, 8 to 10-bit sample bit depth • Main Still Picture Profile 4:2:0 chroma format, 8-bit sample bit depth
©
Ralf Schäfer
8
Image Processing
RD Comparison with MPEG-2 and H.264
Kimono, 1920x1080, 24Hz - Entertainment 42
Y PSNR [dB]
40 38 36
HEVC MP
34
H.264 HP
32
MPEG-2 MP
30 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
bit rate [kbit/s] Results from: [Ohm et al., IEEE TCSVT, 2012] ©
Ralf Schäfer
9
Image Processing
MOS Comparison with H.264 Kimono1, 1920x1080, 24Hz
10
subjective quality (MOS)
9 8
7 6 5
4 3 HEVC MP
2
H.264/MPEG-4 AVC HP
1 0 0
500
1000
1500
2000
2500
3000
bit rate (kbps) Results from: [Ohm et al., IEEE TCSVT, 2012] ©
Ralf Schäfer
10
Fraunhofer Heinrich Hertz Institute
Fraunhofer Heinrich Hertz Institute, Einsteinufer 37, 10587 Berlin
www.hhi.fraunhofer.de
Performance and Complexity
Image Processing
HEVC provides objectively a 35-40% bit-rate reduction on average compared to H.264/AVC HP [Ohm et al., IEEE TCSVT, 2012]
HEVC provides subjectively over 50% bit-rate reduction compared to H.264/AVC HP [Ohm et al., IEEE TCSVT, 2012]
Optimized parallel decoder implementation achieves real-time decoding of 4k @ 60Hz video on 8-core Intel Sandybridge processors [HHI internal study, 2013]
©
Ralf Schäfer
12
Image Processing
Range Extensions Range Extensions: Proposed Timescale • PDAM (Preliminary Draft Amendment) • DAM (Draft Amendment) • FDAM (Final Draft Amendment)
January 2013 July 2013 January 2014
Extended support for • Chroma formats beyond 4:2:0 • Bit depths beyond 10 bits Range Extensions will also support • Lossless coding • Screen content coding
©
Ralf Schäfer
13
Image Processing
Scalable HEVC Extension (1)
Scalable extension of HEVC: • Call for Proposals in July 2012 (responses submitted in October 2012) • First Test Model planed in January 2013 • First Working Draft planned in April 2013 • Finalization planned in July 2014 scalable encoder
5 Mbit/s
AVC/HEVC decoder
10 Mbit/s
scalable decoder
720p
scene
20 Mbit/s
©
Ralf Schäfer
scalable decoder
1080p
(4k)x(2k)
14
Image Processing
3D and Multi-View HEVC Extensions 3DV decoder output format
3DV encoder input format video pictures
video pictures
…
… depth maps camera parameters
…
bitstream
3D video encoder
3D video decoder
… depth maps camera parameters
sender
Multiview video + depth • Camera parameters • Depth is optional
synthesis of intermediate views
bitstream extractor
N-view display 3D receiver
stereo video decoder
stereo receiver
Backwards compatibility • HEVC • Optionally: Stereo video
2D video decoder
Bitstream scalability • View scalability • Optionally: Video only ©
stereo display
2D display 2D receiver
Ralf Schäfer
15
Image Processing
Basic Codec Structure view 0 (independent)
view N-1
view 1
…
HEVC conforming video coder
depth map coder
video coder for dependent views
depth map coder for dependent views
…
video coder for dependent views
depth map coder for dependent views
multiplexer bitstream
HEVC-based codec with additional coding tools for dependent views & depth maps ©
Ralf Schäfer
16
MVC Reference HHI 3D Video Codec
Image Processing
Sequence „Balloons“ (1024x768@30p), synthetised view
MVC, 350 kbps MVC: Multi View Coding (based on H.264)
HHI proposal: 3DVC, 350 kbps, distribution of bitrate: Video1: 72%,Video2: 17%, Depth1: 8%, Depth2: 3%
It will be possible to transmit „autostereoscopic 3D“ at bitrates similar to those for HD using H.264! ©
Ralf Schäfer
17
Market expectations
Image Processing
Several companies are working on HEVC products Several encoder manufacturers showed HEVC demos during IBC 2012
©
and NAB 2013, Many more will be seen at IBC 2013. At IBC HHI will demonstrate real time decoding of 4k @ 60 fps on a PC, a real time SW encoder for full HD and decoders und Window, Linux and MacOS. 4k HEVC decoder chips are announced and prototypes are already demonstarted (e.g. at booth of Deutsche TV-Plattform) In the beginning, HEVC will be mainly used for OTT and mobile video services Later UHD and 3D services will follow HEVC is also a good candidate for the transmission of HD (720p) over DVB-T2.
Ralf Schäfer
18
System Integration of H.265/HEVC
Image Processing
Overview of HEVC system layers
The system interface of HEVC is an important component in the media access chain: Existing transport protocols need new specifications and/or amendments.
©
Ralf Schäfer
19
Transport & Storage of HEVC
Image Processing
Relevant Protocols:
©
MPEG-2 Transport Stream (broadcasting)
RTP (streaming)
ISO Base Media File Format (storage)
MPEG-DASH (HTTP streaming)
Ralf Schäfer
20
Image Processing
Summary
Video Coding is a key technology for the information society and video
©
is dominating the complete Internet traffic Standardized solutions are required to guarantee interoperability as well as fair and reliable licensing conditions High-Efficiency Video Coding (HEVC) is a new standard offering roughly 50% bit rate reduction vs. H.264/AVC HEVC will be used for OTT and mobile video services in the beginning HEVC is also a good solution for the distribution of HD over DVB-T2 HEVC is expected to become the basis for future systems beyond HD such as UHD and autostereoscopic 3D First HEVC products are available in 2013 Standardization of protocols for transport and storage is on the way
Ralf Schäfer
21
Image Processing
Thank you!
©
The Fraunhofer Heinrich Hertz Institute
Ralf Schäfer
Dr. Ralf Schäfer
Einsteinufer 37 10587 Berlin, Germany
Tel: +49 30 31002-560
Mail: ralf.schaefer@ hhi.fraunhofer.de
22
