2004 TV Indexing Index Auto-Making Soccer Video Digests :2005 2 2 3603U043-0 Katsunori Kawaguchi
1 7 1.1.................................... 7 1.2.................................. 7 1.3.................................. 8 2 9 2.1.................................. 9 2.1.1.............................. 9 2.1.2............................ 9 2.1.3.............................. 9 2.2.................................. 11 2.2.1 RGB.............................. 11 2.2.2 CMY.............................. 11 2.2.3 HSV.............................. 12 2.2.4 YCC.............................. 13 2.3.................................. 13 2.3.1................................ 13 2.3.2 2............................... 14 2.3.3................................ 14 2.3.4............................. 14 2.4.................................. 16 2.4.1............................. 16 1
2.4.2................................. 16 2.4.3......................... 17 2.4.4....................... 17 3 19 3.1.......................... 19 3.2....................... 21 3.3 Index......................... 22 4 24 4.1.......................... 25 4.2.......................... 26 5 27 5.1.................................... 27 5.2................................ 28 5.2.1.............................. 28 5.2.2............................ 29 5.2.3.............................. 30 5.2.4......................... 32 5.2.5............................ 32 5.2.6.............................. 33 5.2.7................................ 35 5.2.8................................ 36 5.2.9.............................. 37 5.3................................ 39 5.4............................. 39 6 42 2
6.1................................ 42 6.1.1.............................. 42 6.1.2.............................. 42 6.1.3......................... 43 6.1.4............................ 43 6.1.5.............................. 44 6.1.6................................ 44 6.2................................ 45 7 48 7.1..................................... 48 3
2.1..................... 10 2.2............................... 10 2.3 RGB.................................. 11 2.4 CYM.................................. 11 2.5..................................... 12 2.6..................................... 14 2.7 RGB.............................. 14 2.8.................................... 16 2.9.................................... 17 2.10...................... 18 3.1................................ 20 3.2............................... 20 3.3................... 21 3.4........................ 23 4.1................... 24 4.2.......................... 25 4.3.......................... 26 5.1............................... 29 5.2............................... 30 5.3............................ 32 4
5.4............................. 33 5.5........................... 34 5.6............................ 34 5.7................... 36 5.8................... 36 5.9.................................... 37 5.10................................ 38 5.11.................................. 38 6.1............................... 47 7.1.................................... 49 7.2.......................... 49 5
3.1 Index........................ 23 5.1............................ 32 5.2................................. 35 5.3.................................. 35 5.4 1.............................. 41 6.1............................... 42 6.2............................... 43 6.3............................ 43 6.4............................ 44 6.5....................... 45 6.6....................... 45 6
1 1.1 CS BS TV 1.2 Indexing [1] [4] [5] [6] [7] [8] TV 2 TV TV 7
1 Indexing 1.3 6 2 3 Index 4 5 6 8
2 2.1 2.1.1 x,y 2 (brightness) (gray level value) f(x, y) = g 2.1.2 x,y g x,y T 2.1 2.1.3 2 x,y 2 2 1 9
2 2.1: 1 2 1 2.2 1 1 1 (raster) (raster scan image) 2.2: 10
2 2.2 (color model,color base) 2.2.1 RGB RGB (Red) (Green) (Blue) (additive mixing) 3 R G B 0 255 (8 ) 24 2.3 2.2.2 CMY CYM RGB (cyan) (magenta) (yellow) (subtractive mixing) 3 (black) CMYK 2.4 RGB CMY C = 255 R, M = 255 G, Y = 255 B 2.3: RGB 2.4: CYM 11
2 2.2.3 HSV HSV (Hue angle) (Saturation) (Value) HSV H 0 2π 0 π/3 2π/3 π 4π/3 5π/3 2π 2.5 S V (Value) (Lightness) HSL HLS RGB HSV V = max(r, G, B) { 0 (max(r, G, B) = 0 ) S = 1 min(r, G, B)/ max(r, G, B) ( ) H = 0 (S = 0 ) (G B)/(max(R, G, B) min(r, G, B)) (max(r, G, B) = R ) (B R)/(max(R, G, B) min(r, G, B)) (max(r, G, B) = G ) (R G)/(max(R, G, B) min(r, G, B)) (max(r, G, B) = B ) 2.5: 12
2 2.2.4 YCC YCC NTSC(National Television System Committee) (Y,CIE Y ) (Cb) (Cr) YCbCr YUV YIQ RGB YCC Y = 0.29891 r + 0.58661 g + 0.11448 b Cb = 0.16874 r 0.33126 g + 0.50000 b Cr = 0.50000 r 0.41869 g 0.08131 b 2.3 2.3.1 M N (density histgram) 2.6, 2.7 13
2 2.6: 2.7: RGB 2.3.2 2 1 1 0 2 0 1 2 2 (binary image) 2 2 CPU 2.3.3 2 2 0 1 2 2.3.4 (Affine transformation) 14
2 2 2 2 (x,y) (X,Y) 2 2 2 3 [ ] [ ] X Y W = x y 1 a d 0 b e 0 c f 0 X = ax + by + c, Y = dx + ey + f 3 3 2 3 3 (x,y,z) (X,Y,Z) 3 [ ] [ ] X Y Z W = x y z 1 a e i 0 b f j 0 c g k 0 d h l 0 3 3 2 2 2.8 (x,y,(0)) (X,Y,Z) 3 Z (X,Y ) 4 15
2 [ ] [ ] X Y Z W = x y 0 1 a d p 0 b e q 0 z 1 z 2 z 3 0 c f r 0 X = ax + by + c px + qy + r, Y = dx + ey + f px + qy + r 2.8: 2.4 2.4.1 2.9 2.4.2 16
2! "$# % 2.9: 2.4.3 2.4.4 (pattern matching) (template matching) 17
2 P i = (P i,1, P i,2,..., P i,n ) P T = (P T,1, P T,2,..., P T,n ) D u = n (P i,k P T,k ) 2 k=0!#"$!&%')( * 2.10: ( 2.10 ) 18
3 2 3.1 3.1 Index Index 1. 2. 3.2 19
3 Indexing Index 3. indexing!) * "$#&%('! ;< 8 "$#&%(':8 354 '7698 +,.-0/ 1 2 3.1: 3.2: 20
3 3.2 # #,+ %$&'( -/.103254 "! 67 8:9<;= )%*% 3.3: 3.3 2 1. 2. Indexing 3. Index 4. 1,2 3,4 Index M 1. M S 1, S 2,..., S n T T 1, T 2,..., T n M = {S 1, S 2,..., S n } T = {T 1, T 2,..., T n } 21
3 2. Indexing S i Index I Si 1, I Si 2,..., I Si h S i = {I Si 1, I Si 2,..., I Si h} Index O 1, O 2,..., O l 3. P (S i ) P (S i ) = P ({I Si 1, I Si 2,..., I Si h}) = {I Si 1, I Si 2,..., I Si g,..., I Si h} {O 1, O 2,..., O k,..., O l } IndexI Si g Index O k m t 4. t P (S i ) S 1, S 2,..., S m m X P (S i ) m = {S r1, S r2,..., S ri,..., S rj } { X (k {r1, r 2,..., r j }) P (S k ) < X ( ) r i < r i+1 t j T rk k=1 3.3 Index Indexing Index 3.1 3.4 2 22
3 2 3.1: Index 3.4: 23
4 TV 3 4.1 3 Index TV [1][3] TV 4.1: 24
4 4.1 2 TV TV 4.2 Indexing Indexing Indexing 10% 50% TV 4.2: 25
4 4.2 TV 4.3 Indexing Indexing Index 4.3: 26
5 Indexing TV Microsoft VisualC++ 5.1 (width) 320,360,640,720 dot (height) 240,480 dot 29.97fps 24bit Color VfW(Video for Windows) Avi M F 1, F 2,..., F framemax F i 27
5 5.2 5.2.1 F t F t+1 RGB S(t) k 5.1 M S 1, S 2,..., S n S i F j, F j+1,..., F k j = S i ST, k = S i ED S i ED + 1 = S i+1 ST 255 R(t) = r t+1 (i) r t (i) i=0 S(t) = R(t) + G(t) + B(t) r t (i) : F t r i S(t) k k = 100000 M = {S 1, S 2,..., S n } 28
S i = {F j, F j+1,..., F k } 5 = {F Si ST,..., F Si ED} 5.1: 5.2.2 Y Y < Y cut Y > 255 Y cut Y cut = 40 F t h t h h 20 % H cut i 5.2 h t (i) : F t i Y cut Y 255 Y cut h(i) = framemax t=1 h t (i) 29
/. YX -W 0Z^ H cut = MAX(h(i)) 0.2(0 i 359) 5 h(i) H cut i H feild H feild :? @ A B CDEF 6b 798 c9d : e ; f < g = h > i 254 \5] `5a 0Z^ 132 [3\ _3` 0Z^ G HG IG JG KLG KMG KNG LKG LOG L PG HGG HHG!" H!I"G # #$&%('*)+,?#@#Q&R(S*TUV 5.2: 5.2.3 4.2 3 2 30
5 4.3 H feild H feild F Av F CH 6 60 (i 1) H 60 i(1 i 6) 6 8 H(f) f H Av S ted F Av = h i (j H feild ) /(S t ED S t ST + 1) i=s t ST S t ED 1 F CH = h i (j H feild ) h i+1 (j) /(S t ED S t ST ) i=s t ST S t ED H Av = i=s tst H(i) /(S t ED S t ST + 1) H feild F Av F CH H Av F Av,F CH,H Av k 1,k 2,k 3 5.3 1 : F Av k 1 2 : F Av k 2, F CH < k 3, H Av k 4 2 k 1 = 20000, k 2 = 50000, k 3 = 500, k 4 = 700 1-pass 1 2-pass 2 2 2 1 2 45 5.1 31
5 5.3: 5.1: F Av F CH H Av 1 0 299 1404 139 747 2 300 512 7177 252 2718 3 513 705 23092 383 653 4 706 745 68611 785 551 5 746 776 55996 1534 925....... 5.2.4 5.4 4.3 H feild 2 (r, g, b) = (255, 255, 255) 5.2.5 5.5 3 32
5 5.4: 35 5.6 5.2 4 (h, w) (H, W ) 3 Ah + Bw + 1 Dh + Ew + 1 H = C, W = F P h + Qw + 1 P h + Qw + 1 2 Hough 2 Affine?? 1 2 5.2.6 Y 2 S l e 5.3 e = 4πS l 2 33
5 5.5: 5.6: s 32 32 5.7 5.8 2 34
% $ # "! & ' 5 5.2: H(m) W (m) 1 0 0 2 68 0 3 0 105 4 68 105 5 13.84 0 6 54.16 0 7 13.84 16.5 8 54.16 16.5 9 13.84 88.5 10 54.16 88.5.... 5.3: 5.2.7 5.9 35
5 5.7: 5.8: 2 2 5.2.8 5.10 Indexing Indexing 36
5 5.9: VTR VTR 5.2.9 5.11 [d 1, d 2, d 3, d 4, d 5 ] (d 1, d 2 ) : (H, W ) (d 3, d 4 ) : (h, w) d 5 : 37
5 5.10: 5.11: 38
5 5.3 Index 1. (a) 3m (b) (c) 2. (a) (b) (c) 3. (a) 4. (a) (b) (c) 5.4 5.4 39
5 V s s A s(n) s 1 0 n W (n) W (n) W V s (s = 0, 1,..., scene MAX ) A s(n) = 1 ( n V s ) = 0 ( V t s W (n) = A s(n) W = Av(W (n) ) n Indexing m Indexing 40
5 5.4: 1 int[2] time[2] int int double double double double int double double double double int double double double double CK(IN) CK(OUT) FK(IN) FK(OUT) GK(IN) GK(OUT) SI(IN) SI(OUT) bool bool bool bool bool bool bool bool double GK int double 41
6 6.1 6.1.1 1 90 6.1 6.1: 174 94.6% 10 5.4% 7-6.1.2 1 90 6.2 42
6 50 3 4 6.2: 8 24 0 0 19 16 58 46 20 86.2% 52.2% 80.0% 6.3: 1 2 1 200 74.6% 92.0% 2 200 69.6% 88.6% 5 1 18000 68.6% 87.5% 5 2 18000 65.9% 88.2% 6.1.3 1 90 3 20 100% 6.1.4 6.3 2 2 43
6 6.4: 1 2 90 200 61.0% 67.5% 20 200 53.5% 53.5% 20 200 66.5% 84.0% 20 200 55.3% 68.0% 800 59.6% 68.3% 90 18000 53.9% 62.5% 6.1.5 90 1 20 3 6.4 6 30% 2 6.1.6 5 2 2 1 44
6 6.5: 1 2 200 86.6% 87.5% 200 89.1% 89.1% 400 87.9% 88.3% 90 18000 84.4% 86.2% 6.6: 1 2 200 41.5% 67.0% 200 87.4% 87.6% 400 64.5% 77.3% 90 18000 70.3% 80.5% 2 > 6.5 6.6 2 FK CK FK 4 30% 6.2 Indexing 2 Index 6.1 Index 45
6 Indexing (1) Index Index Index Index (2) Index Index Indexing Indexing (3) 6.1 2 Index Index 46
' & 6 C +,.- (*),.- / "!$#% 02143 57698$: < >@? <?FEHG = =BADC C ; 143 57698$: < >@? <?FEHG I =KJ I =JLA INM ILM ILM 6.1: 47
7 7.1 Index 7.1 7.2 Indexing OPTA [10] 48
7 7.1: 7.2: 49
[1],, 2001,Feb 2002 [2],, 2003,Feb 2004 [3],, 2003,Feb 2004 [4],,,, 106-33 16-33,Feb 2002 [5],,,, TECHNICAL REPORT OF IEICE. PRMU 2000-169,Jan 2001 [6],,,, TECHNICAL REPORT OF IEICE. PRMU 2000-170,Jan 2001 [7],,,, TECHNICAL REPORT OF IEICE. PRMU 2000-171,Jan 2001 [8],,, - -, TECHNICAL REPORT OF IEICE. IE2000-23,PRMU2000-48,MVE2000-52,Jul 2001 [9],V.V.Vinod, - -, D-ll Vol. J81-D-ll No.9,Sep 1998 50
7 [10] http://www.optaindex.com OPTA INDEX 51
OB OB OB M2 M2 3 6 2005 2 52