2012 3 A Graduation Thesis of College of Engineering, Chubu University High Accurate Semantic Segmentation Using Re-labeling Besed on Color Self Similarity Yuko KAKIMI
2400 90% 2 3 [1] Semantic Texton Forests STFs [1] ( ) STFs STFs ColorSelf-Simlarity CSS [2] ii
1 STFs 2 STFs 3 CSS CSS 4 iii
1 1 1.1................ 1 1.2 Semantic Texton Forests............................ 2 1.2.1 Randomized Trees........................... 2 1.2.2............................... 4 1.2.3 Semantic Texton Forests Features................... 4 1.2.4.............................. 5 2 STFs 7 2.1..................................... 7 2.2..................................... 8 2.3....................................... 8 2.4 STFs................................. 11 3 CSS 12 3.1 Color Self-Similarity.............................. 12 3.1.1......................... 12 3.1.2.............................. 13 3.1.3............................. 14 3.2................................ 16 3.2.1......................... 16 3.2.2.................. 16 3.2.3.............................. 17 iv
4 18 4.1..................................... 18 4.2..................................... 18 4.3..................................... 18 4.3.1......................... 19 4.3.2....................... 19 4.4....................................... 20 4.4.1......................... 22 4.4.2....................... 27 28 30 31 v
1.1...................... 3 1.2................................ 4 1.3 Semantic Texton Forests Features....................... 5 2.1................................. 9 2.2 f.................................. 10 2.3 f Confusion Matrix.......................... 11 2.4 CSS STFs......... 11 3.1............................. 13 3.2.................................. 14 3.3 CSS....................... 14 3.4 CSS....................... 15 3.5 CSS...................... 15 3.6 CSS....................... 15 3.7............................ 17 3.8.................................. 17 4.1........................ 20 4.2................................. 21 4.3 f.................................. 22 4.4 f Confusion Matrix.......... 23 4.5 f Confusion Matrix.......... 23 4.6 1 f Confusion Matrix.......... 24 4.7 2 f Confusion Matrix.......... 24 vi
4.8 3 f Confusion Matrix.......... 25 4.9 4 f Confusion Matrix.......... 25 4.10................................ 27 vii
2.1........................ 7 2.2 STFs............................... 8 2.3 f [%].............................. 10 4.1 f [%].............................. 22 4.2 f [%]................................ 26 4.3 [%]............................ 27 viii
1 STFs 1.1 Bag-of-features 3 He CRF [4] Shotton CRF [5] Winn [7] Tu CRF Auto-Context [8] Scroff Texton HOG [9] Shotton Texton Randomized Trees RTs [3] 1
1.2. Semantic Texton Forests STFs[1] Tighe [6] Vezhnevets Geometric Context Multiple Instance Learning Multi-Task Learning [10] 1.2 Semantic Texton Forests Semantic Texton Forests STFs [1] STFs RTs[3] 1.2.1 Randomized Trees Randomized Trees RTs [3] Random Forests Randomized Forests Randomized Trees, Randomized Deicision Forests Randomized Trees RTs 1.1 2
1.2. Semantic Texton Forests 1.1: 3
1.2. Semantic Texton Forests 1.2.2 1.2(a) 1.1 1.2(b) 1.2(b) 1.2: 1.2.3 Semantic Texton Forests Features STFs 1.3(b) d d 4 Semantic Texton Forests Features STFF f(p) = p x,y,c (1.1) f(p) = p x1,y 1,c 1 + p x2,y 2,c 2 (1.2) f(p) = p x1,y 1,c 1 p x2,y 2,c 2 (1.3) f(p) = p x1,y 1,c 1 p x2,y 2,c 2 (1.4) 4
1.2. Semantic Texton Forests (a) Semantic Texton Forests (b) Semantic Texton Forests Features 1.3: Semantic Texton Forests Features p p x, y c CIELab 1 (1.1) CIELab 1.2 1.4 1.3(b) 2 1.2.4 STFs RTs 1 1 1.3(a) STFF f(v) i (1.2) (1.4) t. I l = i I n f(v) i < t (1.5) I r = I n \ I i (1.6) I n n I l I r t f(v) i t E. E = I l I n E(I l) I r I n E(I r) (1.7) 5
1.2. Semantic Texton Forests 1.7. E(I) = n P (c i ) log 2 P (c i ) (1.8) i=1 P (c i ) l D I l I r P (c l) RTs 6
2 STFs STFs grass sky road leaf 4 2.1 QVGA(320 240) 181 92 2.1 2.1: 89 48 92 51 2.2 STFs 7
2.2. 2.2: STFs 8 8 5 10 400 5 0.25 2.2 = (2.1) = (2.2) 2 f f 2.3 f = 2 1 + 1 = 2 + (2.3) Confusion Matrix 2.3 4.2 8
2.3. 2.1: 9
2.3. f 2.2 2.3 f Confusion Matrix 2.3 Confusion Matrix 2.2: f 2.3: f [%] grass sky road leaf STFs 88.4 87.6 94.1 79.0 87.3 STFs 83.3 93.8 67.1 66.2 77.6 2.2 2.3 f 75% 4.2 10
2.4. STFs (a) Confusion Matrix (b) Confusion Matrix 2.4 STFs 2.3: f Confusion Matrix STFs 2.4(a) STFs CSS 2.4(b) 2.4: CSS STFs 11
3 CSS CSS CSS 3.1 Color Self-Similarity Color Self-Similarity(CSS) [2] 2 HSV (H: S: V: ) 3 3.1.1 3.1 d d HSV 3 b = 3 3 12
3.1. Color Self-Similarity 3.1: 3.1.2 P 3.1 f = 3 3 (Pb 1 P b 2)2 (3.1) b=1 P b b f S.Walk 13
3.1. Color Self-Similarity 3.2: 3.1.3 CSS 2 CSS CSS 1 ) 3.3 3.6 3.3: CSS 14
3.1. Color Self-Similarity 図 3.4: CSS の可視化 観測パッチ 葉 図 3.5: CSS の可視化 観測パッチ 芝生 図 3.6: CSS の可視化 観測パッチ 空 15
3.2. CSS 3.2 STFs CSS 3.2.1 CSS 3.3 3.6 HSV 1 n = 1,..., N CSS n 3.3 3.6 QVGA 320 240 8 8 1200 n 1200 3.2.2 3.7 n i = 1,..., N h i (y) 3.2 h i (y) n STFs y P (y n) i CSS s(i, n) 2 r N h i (y) = s(i, n)p (y n)ω(r)δ[y, STFs(n)] (3.2) n=1 ω STFs(n) n STFs δ[ ] 2 1 0 16
3.2. 3.7: 3.2.3 3.8 h i (y) y i 3.3 y i = argmax h i (y) (3.3) y Y 3.8: 17
4 2 4.1 STFs 2.1 4.2 STFs 4.3 18
4.3. 4.3.1 2.2 2.3 4.4.1 f f road leaf 4.1 4.3.2 4.1 4 1 4.1 = void (4.1) 19
4.4. 4.1: 4.4 4.2 20
4.4. 4.2: 21
4.4. 4.4.1 f 4.3 4.1 f Confusion Matrix 4.4 4.5 4.3: f 4.1: f [%] grass sky road leaf STFs 88.4 87.6 94.1 79.0 87.3 83.5 60.4 90.5 73.9 77.1 STFs 83.3 93.8 67.1 66.2 77.6 77.0 73.0 65.6 60.4 69.0 22
4.4. (a) STFs (b) 4.4: f Confusion Matrix (a) STFs (b) 4.5: f Confusion Matrix 23
4.4. 4.3 4.5 4.1 f STFs 4.2 f Confusion Matrix 4.6 4.9 f 4.2 (a) STFs (b) 4.6: 1 f Confusion Matrix (a) STFs (b) 4.7: 2 f Confusion Matrix 24
4.4. (a) STFs (b) 4.8: 3 f Confusion Matrix (a) STFs (b) 4.9: 4 f Confusion Matrix 25
4.4. 4.2: f [%] STFs 1 98.5 91.1 2 91.3 87.6 3 94.5 85.9 4 88.1 79.6 4.6 4.9 4.2 STFs void STFs 26
4.4. 4.4.2 4.10 4.3 4.10: 4.3: [%] grass sky road leaf STFs 59.9 55.6 36.5 33.1 46.3 91.3 47.1 85.2 62.1 71.4 STFs 74.6 71.1 26.9 26.9 49.9 94.4 51.5 39.6 43.1 53.2 4.10 4.3 STFs 25.1% 7.3% 27
STFs CSS 1 STFs 2 STFs f 75% 3 CSS CSS 4 STFs STFs f STFs STFs void STFs STFs 25.1% 7.3% 28
CSS 29
30
[1] J. Shotton, M. Johnson and R. Cipolla. Semantic Texton Forests for Image Categorization and Segmentation. In Proc. Computer Vision and Pattern Recognition, pp. 1.8, 2008. [2] S.Walk and N.Majer New Features and Insights for Pedestrian Detection, IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2010. [3] L.Breiman Random forests, Machine learning, 45, 1, pp. 5.32 2001. [4] X.He, R.S.Zemel, and M.A. Carreira-Perpinan, Multiscaleconditional random fileds for image labeling, Computer Vision and Pattern Recognition, 2004. [5] J.Shotton, J.Winn, C. Rother, and A. Criminisi, Textonboost:Joint appearance, shape and context modeling for multi-class object recognition and segmentation, European Conference on Computer Vision, 2006. [6] J.Tighe, and S.Lazebnik, Superparsing: Scalable nonparametric image parsing with superpixels, European Conference on Computer Vision, 2010. [7] J.Winn, and J. Shotton, The layout consistent random field for recognizing and segmenting partially occluded objects, Computer Vision and Pattern Recognition, 2006. [8] Z.Tu, Auto-context and its application to high-level vision tasks, Computer Vision and Pattern Recognition, 2008. [9] F.Schroff, A.Criminisi, and A.Zisserman, Object class segmentation using random forests, British Machine Vision Conference, 2008. 31
[10] A.Vezhnevets, and J.M.Buhmann, Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning, Computer Vision and Pattern Recognition, pp.3249-3256, 2010. 32
( ) 2012 3