28 Information selectivity in mixed reality environment 1170317 2017 3 7
AR AR AR AR AR HMD AR AR AR AR AR i
AR ii
Abstract Information selectivity in mixed reality environment Yoshitaka Sakamoto In the means of obtaining information of the outside world, the visual information occupies much. However, it is impossible to process all of the information projecting to the eyes. Since parallel processing of the visual system has limitations, it is necessary to select useful information within the limit. In the information selection, visual attention plays an important role. With the recent increase in the use of smart-phones, many applications using augmented reality (AR) technology appeared and AR has become more familiar. As AR technology develops, it is conceivable that more information by the AR will be additionally presented in the real world. At that time, we need to select useful information from the mixed reality environment where real objects and AR objects are presented. In this study, we constructed a mixed reality environment using head mounted display and examined the property of attention in information selection. A flanker task paradigm was used in this study. The task requires the observers to identify the target stimulus while ignoring the peripheral stimulus. In this paradigm, the difference of reaction time between the condition in which the target and peripheral stimulus was different and the conditions in which these stimuli were same, are considered to be the effect of the peripheral stimulus (flanker effect). When this value is small, the target stimulus is considered to be not influenced much by the peripheral stimulus and the target stimulus can be identified efficiently. The results showed different effects of the flanker on the first and second day in the iii
mixed reality condition in which real objects and AR objects are combined. On the first day, the condition that the target stimulus was a AR object and the flankers were real objects showed smaller flanker effect while on the second day, the condition that the target stimulus was a real object and flankers were AR objects showed smaller flanker effect. The real objects and the AR objects have different characteristics of reflection and shadows due to different illumination light. On the first day, when the real object was the target stimulus, the reaction time might be delayed as attention was captured to AR objects in peripheral, but on the second day the obstruction effect by the flankers decreased. Compared with the first day, the reaction time became faster. In addition, the effect of the day was also found in the condition that both the target and the flanker were both AR objects which had chromatic color. Unlike the condition of mixed reality, the effect of reaction time due to the day was also observed in the reaction time of the condition under which the target stimulus and the peripheral stimuli were same. On the first day, the object with high chromatic color saliency captured attention to the object itself and it might be hard to access the information of the symbol inside the object. key words Visual AttentionARMixed RealityInformation SelecivityFlanker Task iv
1 1 1.1...................................... 1 1.2...................................... 2 1.3 AR(Augmented Reality).......................... 2 1.4 VR(Virtual Reality)............................. 4 1.5 MR(Mixed Reality)............................. 5 1.6.................................. 6 1.7................................ 6 1.8 Ovrvision Pro................................ 9 2 10 2.1................................... 10 2.1.1......................... 10 2.1.2............................ 11 2.1.3................................ 13 3 14 3.1................................... 14 3.1.1................................. 14 3.1.2......................... 14 3.1.3................................. 19 3.2................................... 21 3.2.1............................. 21 3.2.2......................... 23 v
3.2.3 AR.................. 28 4 29 4.1................................ 29 4.2 AR ()......................... 29 5 31 32 33 vi
1.1 AR.......................... 3 1.2 VR................................ 4 1.3 Microsoft HoloLens([5])........................... 5 1.4 ARAVVRMR (Milgrame et al. (1994) [6] )..................................... 5 1.5................................ 8 1.6 Ovrvision Caliblation Tool.......................... 9 2.1 ARUco............................... 11 2.2............................... 11 2.3...................... 11 2.4 HMDOculus Rift DK2.......................... 11 2.5 Ovrvision Pro................................. 12 2.6 OvrvisionProhuzo HMD...................... 12 2.7............................ 13 3.1 AR - R 1....................... 15 3.2 AR - R 2....................... 15 3.3 AR - R 1...................... 15 3.4 AR - R 2...................... 15 3.5 R - AR 1....................... 16 3.6 R - AR 2....................... 16 3.7 R - AR 1...................... 16 3.8 R - AR 2...................... 16 vii
3.9 R - R 1........................ 16 3.10 R - R 2........................ 16 3.11 R - R 1....................... 17 3.12 R - R 2....................... 17 3.13 AR - AR 1...................... 17 3.14 AR - AR 2...................... 17 3.15 AR - AR 1..................... 17 3.16 AR - AR 2..................... 17 3.17 AR - AR 1..................... 18 3.18 AR - AR 2..................... 18 3.19 AR - AR 1.................... 18 3.20 AR - AR 2.................... 18 3.21................................ 19 3.22................................... 19 3.23 HMD......................... 20 3.24............................ 21 3.25.......... 22 3.26....................... 24 3.27.......................... 24 3.28 A........................ 25 3.29 B........................ 25 3.30 C........................ 26 3.31 D........................ 26 3.32 E........................ 26 3.33 F........................ 26 3.34 G........................ 27 viii
3.35 H........................ 27 3.36 I........................ 27 3.37 J........................ 27 3.38 AR - AR............ 28 ix
1 1.1 2012 500 2018 2700 [1] AR(Augmented Reality) AR AR 1.2 AR CG 2016 AR Pokemon GO Pokemon GO AR Pokemon GO 6 [2] AR AR AR AR AR MR : Mixed Reality) MR 1.4 MR CG 1
1.2 AR 1.2 ( 1.5 ) AR 1.6 [9] 1.3 AR(Augmented Reality) AR(Augmented Reality) AR GPS AR GPS 2
1.3 AR(Augmented Reality) などの情報を取得し 付加情報を合成する座標が決定する GPS や各センサーはスマート フォンやタブレットに標準で搭載されているので比較的簡単に実現できるが GPS 精度の 限界から表示のズレが起こるデメリットがある マーカー型は AR マーカーという画像解析 を容易にするマーカーを用いて現実世界と表示させる AR 物体の座標をリンクさせる方式 である 付加情報の表示位置を正確に指示できるのが最大のメリットである しかし状況に 応じて必要な数だけマーカーを準備しなくてはいけないなどのデメリットもある マーカー レス型は AR マーカーを使わずに現実世界の物体や空間全体を把握することで付加情報の 表示位置を決定する方式である 大きな対象に対してもピンポイントで付加情報を合成でき るが 高度な画像処理が必要で技術的なハードルは高い [3] 本実験ではマーカー型 AR を 使用した 使用したマーカーは ARUco という BSD ライセンスで提供されている OpenCV をベースにした AR ライブラリである ARUco を用いて実際にオブジェクトを呈示した例 を図 1.1 に示す AR 技術は昨年ヒットを起こした Pokmon GO やスマートフォンのカメラ を用いてナビゲーションを行う道順案内アプリ [4] 医療分野における手術支援システムで ある ダヴィンチ で使われるなど幅広い分野で使用されている 図 1.1 AR のオブジェクト呈示例 3
1.4 VR(Virtual Reality) 1.4 VR(Virtual Reality) VR(Virtual Reality) VR 3 3 VR VR VR [3] HTC Valve Corporation HTC VIVE VR HMD VR HTC VIVE 1.2 1.2 VR 4
1.5 MR(Mixed Reality) 1.5 MR(Mixed Reality) MRMixedReality) CG VRReality Microsoft MR HoloLens 1.3 AR (AV : Augmented Virtual) 1.4 ARAVVRMR 1.4 AR AV MR [7] 1.3 Microsoft HoloLens([5]) 1.4 ARAVVRMR (Milgrame et al. (1994) [6] ) 5
1.6 1.6 (Posner & Cohen, 1984) [8] 1.7 1974 Barbara A. Eriksen Charles W. Eriksen HHHHH SSHSS [9] 6
1.7 ( 1.5) Eriksen & Eriksen1974 H GJQHGJQ NWZHNWZ [10] [11] AR AR AR 7
1.7 1.5 8
1.8 Ovrvision Pro 1.8 Ovrvision Pro Ovrvision Pro Oculus Oculus Rift HTC VIVE AR.com Wizapply EEPROM GPIO.Unity Ovrvision Pro SDK Ovrvision Pro SDK MIT [12] Ovrvision SDK Ovrvision Cliblation Tool ( 1.6) Ovrvision Pro 2.1.2 1.6 Ovrvision Caliblation Tool 9
2 2.1 AR PC OS : Windows8.1 CPU : Intel Core i5-4460 3.2GHz : 8GB 2.1.1 Windows Visual Studio 2015 Unity5.3.6p2(64bit) Microsoft Excel 2013 R (3.3.2) 10
2.1 2.1.2 AR AR ARUco ( 2.1) 4.5cm ( 2.2) 8cm 60cm 3cm ( 2.3) (HMD) OculusRift DK2( 2.4) AR Ovrvision Pro ( 2.5) Ovrvision Pro HMD 2.6 2.2 2.1 ARUco 2.3 2.4 HMDOculus Rift DK2 11
2.1 2.5 Ovrvision Pro 2.6 OvrvisionProhuzo HMD 12
2.1 2.1.3 (ERE- COM TK-TCM011BK) ( 2.7) 2.7 13
3 3.1 5 AR 3.1.1 10 3.1.2 <<<<<>>>>> <<><<>><>> AR AR- R ( 3.1 3.4) AR R- AR ( 3.5 3.8) 14
3.1 R- R ( 3.9 3.12) AR AR- AR ( 3.13 3.16) 4 AR ( AR - AR) AR - AR ( 3.17 3.20)AR x,y,z = 0.1 x,y,z = 0 x,y,z = 0,0.1,-0.1 AR - AR RGB 44.224.230 3.1 AR - R 1 3.2 AR - R 2 3.3 AR - R 1 3.4 AR - R 2 15
3.1 3.5 R - AR 1 3.6 R - AR 2 3.7 R - AR 1 3.8 R - AR 2 3.9 R - R 1 3.10 R - R 2 16
3.1 3.11 R - R 1 3.12 R - R 2 3.13 AR - AR 1 3.14 AR - AR 2 3.15 AR - AR 1 3.16 AR - AR 2 17
3.1 3.17 AR - AR 1 3.18 AR - AR 2 3.19 AR - AR 1 3.20 AR - AR 2 18
3.1 3.1.3 AR 50cm ( 3.21) Ovrvision Pro HMD AR HMD 3.22 3.21 3.22 HMD 30cm AR 3.23 300ms 3.1 3.20 20 2 40 1 10 3.1 3.20 20 1 20 19
3.1 HMD 3.23 AR AR AR 3.23 HMD 20
3.2 3.2 3.2.1 (SD) SD 2 AR - AR 3.21 3.22 3.24 1 (F (1,9) = 1.88p = 0.14 3.25 2 F (1,9) = 2.86, p < 0.05 R - AR AR - AR p < 0.01p < 0.05 3.24 21
3.2 3.25 22
3.2 3.2.2 2 F (1,9) = 10.586, p < 0.01 3.26 R - AR AR - R R - AR 3.25 AR - R 3.27 2 R - AR p < 0.01 AR - R R - AR R- AR, AR AR AR 4.1 23
3.2 3.26 3.27 24
3.2 1 3.28 3.37 3.30 3.31 3.35 3 AR - R R - AR 3.33 3.36 3.37 3 AR - R, AR 3.32 AR - R 3.29 3.34 3.28 R - AR AR - R 3.28 A 3.29 B 25
3.2 3.30 C 3.31 D 3.32 E 3.33 F 26
3.2 3.34 G 3.35 H 3.36 I 3.37 J 27
3.2 3.2.3 AR AR - AR AR - AR 3.38 AR - AR AR - AR 2 F (1,9) = 7.48p < 0.05p < 0.05p < 0.01 3.38 AR - AR 28
4 4.1 AR AR R - AR AR - R AR AR AR AR AR - R AR 4.2 AR () AR - AR R - AR AR AR - AR 29
4.2 AR () AR 2 AR - AR AR 30
5 HMD AR AR AR AR AR AR 31
,,.,,.,.. 32
[1] 27 http://www.soumu.go.jp/johotsusintokei/whitepaper/ja/h27/html 2017 1 6 [2] App Annie 2016 3 https://www.appannie.com/jp2017 2 8 [3] VRAR pp20-322015 [4], 2010 [5] Microsoft HoloLens https://www.microsoft.com/microsoft-hololens/ja-jp 2017 2 10 [6] Paul MilgramHaruo TakemuraAkira UtsumiFumio Kishimo Augmented Reality: A class of displays on the reality-virtuality continuum Telemanipulator and Telepresence Technorogies1994 - SPIE Vol. 2351282-292,1994 [7] 1998 [8] Cognitive Psychology : Brain, Modeling and Evidence pp66-932010 [9] Eriksen,B.A.& Eriksen,C.W.Effect of noise letters upon the identification of a target letter in a nonsearch task,p ercept& P sychophysics,143-1491974 [10] Vol. 82141-1492011 [11] Ridderinkhof,K.R. & Band,G.P.& Logan,D. A study of adaptive behavior: effects of age and irrelevant information on the ability to inhibit one s actions. Acta 33
psychologica. 101: 3153371999 [12] Ovrvision Pro USB3.0 Open-Source Stereo Camerahttp://ovrvision.com/ 2017 2 7 34