(1) - E-Mail: katto@waseda.jp
Y U V R G B (1/30 ) RGB / YUV = B G R V U Y 0.31 0.52 0.21 0.32 0.28 0.60 0.11 0.59 0.30 RGB YUV
CCIR 601 4:4:4 4:2:2 4:2:0 Y Y Y U V U V U V YUVUV UV 4:2:0 4:2:2
(RGB8 ) TV TV HDTV 352x240 720x480 1920x1080 21Mbit/s 83Mbit/s 498Mbit/s
MC+DCT (20 ) DCT DCT + MC ( motion compensation) DCT ( discrete cosine transform) Q:
DCT +
DCT ( : 0.9 ) DCT ( ) ( )
DCT ρ=0.95 KLT: DCT: KLT 8x8 DCT
Wavelet ( ) x(n) h 0 (n) 2 h 0 (n) h 1 (n) 2 2 h 0 (n) h 1 (n) 2 2 LLL LLH LH h 1 (n) 2 H 2 LLL LLH LH H LL 0 π LH H
DCT Wavelet ρ=0.95 Wavelet DCT ( ) Wavelet: (JPEG-2000)
IP I P P P P P P I: I ( ) ( : 0.9 ) P: P ( )
IPB I B B P B B P I: I ( ) P: P ( ) ( ) B: B ( )
I B B P B B P TV (MPEG-2) DCT: DCT
() 1/4 1/8 : 0.5 0.5 0.5 0.25
:
H.261 1990 64kb/s2Mb/s ISDN ITU-T H.263 1996 kb/s H.263+ 1998 kb/s H.264 2003 kb/s MPEG-1 1992 1.5Mb/s CD-ROM ISO MPEG-2 * 1995 Mb/s MPEG-4 1999 kb/s * MPEG-2/H.262H.264 (MPEG-4 Part 10) ISO ITU-T
82 84 86 88 90 92 94 96 98 00 02 MC DCT ( ) MC+DCT B ITU-T H.120 Sub-rate H.261 H.263 / H.263+ 1.5M 384K 64K - 1.5M ISDN, Internet, Joint (ITU-T & ISO) MPEG-2 / H.262 DVDHDTV H.264 Internet, ISO MPEG-1 MPEG-4 CD-ROM Internet,
MC+DCT 1/2IPB H.261 - - - - - - H.263 - - - - MPEG-1 - - - MPEG-2 - H.263+ - MPEG-4 H.264 - + ( )
ISO/IEC MPEG-4
MPEG-4 ( ) ( ) ( ) (Wavelet )
MPEG-4 ( ) Hello Hello () ( )
(CIF, QCIF, ) (x, y) VOP (w, h)
MB (1) (2) (3) MB (1) I- DCT (2) P-0 DCT
(1) 2 (0,1) (2) (0, 255) (DCT) ( ) (R, G, B, A) (Y, U, V, A) A: ( ) A = 0: (transparent) A = 255: (opaque) A = 1 254: ( )
ITU-T H.264 (MPEG-4 Part 10)
H.264 (MPEG-4 70%) ( ) (Sub-macroblock prediction) 1/4 1/8 (1/4, 1/8 pel interpolation) (Multiframe prediction) (SAT: sum of absolute transformed differences) (Adaptive intra prediction) DCT (Adaptive block-size transform) CABAC (Context-based Adaptive Binary Arithmetic Coding) VCL: Video Coding Layer NAL: Network Adaptation Layer RTP Packetization
mode 1 16x16 block 1 vector mode 2 8x16 block 2 vectors mode 3 16x8 block 2 vectors mode 4 8x8 block 4 vectors 0 0 1 0 0 1 1 2 3 mode 5 4x8 block 8 vectors mode 6 8x4 block 8 vectors mode 7 4x4 block 16 vectors 0 1 2 3 0 2 1 3 0 1 2 3 4 5 6 7 4 5 6 7 4 6 5 7 8 9 10 11 12 13 14 15 Sub-macroblock prediction
P P P P P P Multi-frame prediction
SAT ( ( SAD minimization ( ) DCT SAD minimization (SAD DCT ) SAT minimization ( ) DCT SAT minimization ( ) SAD: sum of absolute differences, SAT: sum of absolute transformed differences
I A B C D E a b c d F e f g h G i j k l H m n o p mode 0: DC prediction mode 1: Vertical/Diagonal prediction mode 2: Vertical prediction mode 3: Diagonal prediction mode 4: Horizontal prediction mode 5: Horizontal/Diagonal prediction I A B C D E a b c d F e f g h 5 G i j k l H m n o p 4 1 2 3 mode 0: = (A+B+C+D+E+F+G+H) / 8 mode 1: a = (A+B) / 2 e = B b = i = (B+C) / 2 f = m = C c = j = (C+D) / 2 d = g = h = k = l = n = o = p = D
CABAC (Binarization) A B C Binarization (unary) index 0 1 binarization 1 01 A, B: 2 001 3 0001 4 00001 C: 5 000001 vs. ( ) Context-based Adaptive Binary Arithmetic Coding
( CABAC 1015% 5% 5% 5% R-D ( ) 1015% 30%
Rate-Distortion Optimization Multiple Distortion Coding Wyner-Zip Coding
R-D Lagrange (Rate-Constrained minimize J = D + λ R (D, R) ( ) INTRA/INTER ( ) INTRA/INTER/SKIP ( ) ( ) (FastStreaming) Rate-Distortion Optimization for Video Compression, IEEE Signal Processing Magazine, Nov.1998.
R-D (DCT ) (1) J dj dr dd = dr + λ λ MODE = dd dr (2) b R( D) = a log2 D 2 (2Q) D = = 12 Q 3 2 (H.263) MODE (3) λ = c Q 2 MODE c 2 Q 3 λ = ln 2 c = 3 a 0.85 (H.264)
R-D RD D motion min J motion = Dmotion + λmotion Rmotion min ( ) D mode min J mode = Dmode + λmode Rmode min ( )
Multiple Description Coding ( ) () Temporal scalability: I B B P B B Multiple Description Coding: 1 3 2 4 (descriptions)
Wyner-Ziv Coding Input (X) encode decode Output (Y) Distributed Source Coding (Wyner-Ziv Coding): Input (X) encode decode Y (= X+) p subencode subdecode Reconstruction Output (Y ) MAP Y ' = arg max P( X Y, p) p (sub-information)
H.264 LSI Logic, VLE4000: Sand Video (Broadcom), SV-D011: H.264 SDTV encoder (FPGA) H.264 HDTV decoder (core) H.264 SDTV decoder: NECELMP211 (DSP & 3CPU/ARM) Sony: PSP (HW core)cell RenesasSH-Mobile4 (HW core)
Qualcomm: MSM7000 NTT DoCoMo 1 2 (OMAP, SH-Mobile) RF-CMOS Jul.2004 2G/3G WLAN ( ) Brew 1 3D-CG (ATI core) + TI (OMAP ext.) Renesas (SH-Mobile ext.)
ViXS Systems Inc. (MPEG & 802.11) LSI XCode II Matrix ABFM ( ) MPEG-2/4 LSI SDTV 4 8 HDTV 1 ( ) HDD IEEE 802.11a/e LSI 802.11a/e 2 (54 * 2 = 108Mb/s) LAN HDTV MAC ( ) AV (e.g. HDD to ) (Adaptive Bandwidth Footprint Management) Matrix XCode II LAN