THE INSTITUTE OF ELECTRONICS INFORMATION AND COMMUNICATION ENGINEERS TECHNICAL REPORT OF IEICE. 619 289 3 5 66 851 E-mail: {j-satakeakihiro-k}@nict.go.jp {hirayamakawashimatm}@i.kyoto-u.ac.jp UXGA 3fps Abstract Accuracy Improvement of Real-Time Gaze Estimation using High Resolution Camera Junji SATAKE Akihiro KOBAYASHI Takatsugu HIRAYAMA Hiroaki KAWASHIMA and Takashi MATSUYAMA National Institute of Information and Communication Technology 3 5 Hikaridai Seika-cgo Soraku-gun Kyoto 619-289 Japan Kyoto University Yoshida-Honmachi Sakyo-ku Kyoto 66 851 Japan E-mail: {j-satakeakihiro-k}@nict.go.jp {hirayamakawashimatm}@i.kyoto-u.ac.jp We develop an interactive information display that estimates face and gaze directions of a user in real-time and interactively controls contents by recognizing the user s interest and reaction. The gaze estimation method that uses only the camera image generally estimates the gaze direction as a line that connects the center and the eyeball center. However because the resolution of eye area was low the influence of error was large. We report an accuracy improvement of real-time gaze estimation using high resolution camera and detailed model. Key words gaze estimation high resolution camera detection 1. [1] [2] 1 2. 2. 1 1
5 インチ PDP カメラ (a) (b) 1 Fig. 1 Information display. (a) f=25mm (b) f=16mm 2 UXGA Fig. 2 Examples of captured image. [3] [4] 1 [5] [7] 1 3pixel 5 1 7pixel [8] LmedS [9] 5% 2. 2 1(a) 1(b) Point GreyResearch Grasshopper UXGA (16 12) 8bit 3fps 1 / 1.8 CCD IEEE-1394b FUJINON HF25HA-1B f=25mm HF16HA-1B f=16mm 2 6pixel 3 Fig. 3 Expansion image of eye part. RIFA-F 5 5cm 3 6pixel 2pixel [3]. 2. 3 [3] [1] [11] 4 mm.2.6mm.2mm 11mm 2
23 24mm 13mm 1 5 α l l = c c 2 r 2 sin α r = 5.5mm c = 5.8mm d = 13mm 3. 4 α α < arccos r c 18 3. 3. 1 6 1m Z w X w Y w Z c [ ] T [ λ u v 1 = P X w Y w Z w 1 P OpenCV 3. 2 Active Appearance Model(AAM) [12] [13] AAM 45 7 ] T 固視点視軸光軸 黒く見える領域 カメラ 曲率半径 c 虹彩中心 Fig. 4 虹彩半径 r d 左目を上から見た図 角膜 角膜による屈折 虹彩 ( 直径 11) 4 強膜 5.5 7.7 角膜曲率中心 水晶体 鼻側 11.5 眼球 ( 直径 23) 眼球中心 ( 旋回中心 ) 中心窩 耳側 Structural model of eyeball. 旋回中心 (a) Fig. 6 黒く見える領域 5 Fig. 5 3. 3 光軸 厚み l 図 (c) の視点方向 虹彩中心 (b) 接点厚み l 虹彩中心曲率中心 (c) α Calculation of outline. カメラ座標系モデル座標系 6 Xc Zc Xw Zw Yw ワールド座標系 r c 画像座標系 u v Yc Xm Zm Ym Definition of each coordinate system. 6 3 3 [ ] T = ϕ θ ψ t x t y t z i (X mi Y mi Z mi ) (X wi Y wi Z wi ) X wi Y wi Z wi = R z(ϕ)r y(θ)r x(ψ) X mi Y mi Z mi + t x t y t z 3
Fig. 7 7 AMM Example of Active Appearance Model. feature point direction of face turn center of eye Zm [mm] feature point direction of face turn center of eye (a) -4 4 Zm [mm] 4-4 -8-4 Xm [mm] Ym [mm] -4 4-4 4 8 8 4-4 Xm [mm] 8 3 Fig. 8 3D face shape model. 4 Ym [mm] (b) (c) (d) 9 Fig. 9 Iris detection. f AAM (u i v i ) (u i ( ) v i ( )) k+1 = k a f ( k) f( ) = w i {ui ( ) u i } 2 + {v i ( ) v i } 2 i a w i AAM 45 15 9(a) 5 4 w i = 1 11 3 8 3 3 4 3 d 13mm 3. 4 9(a) 3. 2 9(b) [14] 9(c) 2. 3 3. 3 9(d) ( X Fig. 1 Y Z ( u ) v 1 ) d ( X center Y center Z center Estimation of center and gaze. 1.2 I src I temp E ( 1 < = E < = 1) 1 1 I src = 1 I temp = 1 E (u v) = 3. 5 Isrc (u + u v + v ) I temp (u v ) {Itemp (u v )} 2 3 (X center Y center Z center ) 3. 3 d (u v ) 3 (X Y Z ) 1 X u Y λ v = P Z 1 1 X Y Z ) X center Y center Z center = d 2 (X Y Z ) 4
X gaze Y gaze X gaze Y gaze Z gaze X center Y center Z center = λ X Y Z X center Y center Z center 1m Z gaze = 1 3. 6 3. 5 6 3 (X gaze Y gaze ) E E >.4 X gaze = E X gaze E Y gaze = 4. 4. 1 E Y gaze E 2cm 15 11 1m 12 12 7 8 9 2.2pixel 33mm α 7 8 9 1 2 3 α > 3 UXGA 3 1 1 4 3 6 9 1 2 3 4 5 6 左カメラ 7 8 9 右カメラ 1 11 12 13 14 15 下カメラ (a) (b) 11 Fig. 11 Experimental environment. 7 8 9 (a) (b) 12 Fig. 12 Results of detection. 15mm/pixel E XGA 1 78.3mm UXGA XGA 25mm UXGA 3 52.9mm 3 1mm 4. 2 PC Intel Core2 Duo 2.66GHz 2GB 1 PC 2 Intel Integrated Performance Primitives OpenCV cvremap 5
1 [mm] 5 Table 1 Error of gaze estimation. UXGA XGA 1 89.1 13.3 68.5 38.2 64.1 24.4 48.1 29.5 96.1 64.4 38.3 32.5 2 63.6 64.7 23.1 12.8 78.3 7.3 39.9 31. 86.4 83.3 47.2 36.5 3 19.9 59.8 37. 44.9 114.2 25.7 39. 35.1 113.7 92.4 55.9 72.3 4 1.1 66.3 59.3 46.1 95.8 84.8 64.7 61.7 112.1 98.2 79.9 72.8 5 93.5 92.3 46.2 34.3 12.5 17.5 54.6 46.1 43.5 185. 59.6 48.1 6 71.5 42.3 17.1 28.8 92.2 32.2 13.3 43.2 18.3 82.7 35.7 57.9 7 95.5 18.6 62.3 72.5 98.4 125.1 74.1 85.2 167.4 133.2 113.8 97.1 8 82.5 54.4 32.9 39.1 89.7 77.3 42.1 46.7 93.1 35.5 59.6 3.9 9 116. 83.9 46.5 7.3 13.6 78. 49. 78.5 179.1 17.9 68.5 11.6 1 6.5 6.5 71.3 71.3 132.5 132.5 11 44.8 44.8 56.3 56.3 26.5 26.5 12 19. 19. 25.4 25.4 99.9 99.9 13 13.6 13.6 11.3 11.3 116.6 116.6 14 119.7 119.7 128.1 128.1 164.5 164.5 15 59. 59. 65.7 65.7 76.3 76.3 91.3 65. 53.3 52.9 96.2 69.5 58.8 6.9 111.1 15.1 78.3 78.3 UXGA 3fps 1ms 5. UXGA 3 3 3fps No.184946 [1] HCI no.99 pp.9 16 27. [2] Mindprobing: HCI no.99 pp.1 8 27. [3] 2 vol.44 no.4 pp.1136 1149 23. 2 Table 2 1 Time of processing a frame. UXGA XGA.8ms.5ms 11ms 4.5ms 15ms 1ms 3ms 3ms 1.3ms 1ms (1ms) (7ms) 31ms 19ms [4] C. Hennessey B. Noureddin and P. Lawrence A Single Camera Eye-Gaze Tracking System with Free Head Motion ETRA pp.87 94 26. [5] R. Newman Y. Matsumoto S. Rougeaux and A. Zelinsky Real-Time Stereo Tracking for Head Pose and Gaze Estimation FG pp.122 128 2. [6] CVIM vol.47 no.sig15 pp.1 21 26. [7] MIRU pp.319 324 27. [8] J.G. Wang E. Sung and R. Venkateswarlu Estimating the eye gaze from one eye CVIU vol.98 pp.83 13 25. [9] LMedS MIRU pp.i-684-689 24. [1] O plus E vol.22 no.4 pp.418 43 2. [11] http://www.ocular.net/jiten/ [12] T.F. Cootes G.J. Edwards and C.J. Taylor Active appearance models PAMI vol.23 no.6 pp.681 684 21. [13] M.B. Stegmann B.K. Ersbøll R. Larsen FAME - A Flexible Appearance Modeling Environment IEEE Trans. Medical Imaging vol.22 no.1 pp.1319 1331 23. [14] (D) vol.63 no.4 pp.349 356 198. 6