27 MixedReality

Size: px

Start display at page:

Download "27 MixedReality"

えのかたいわ
5 years ago
Views:

1 修士論文の和文要旨研究科専攻大学院情報理工学研究科情報通信工学専攻博士前期課程氏名大和田瑛美華学籍番号論文題目 MixedReality 空間における仮想書斎システムの開発要旨本研究では,MR(MixedReality, 複合現実 ) 空間を用いた仮想書斎システムを開発した. 本システム内では, 電子書籍の閲覧を行ったり, 壁に写真を飾ったりすることが出来る. また, ユーザ自身が本当に歩くことによって仮想空間内を移動したり, 手で触る操作をすることでページをめくったり物を移動したりすることが出来る. 近年電子書籍が普及しているものの, ページの厚みにより読んだ量や残りの量が分かり易いページを捲 ( めく ) ることで読んでいる実感があるなど紙書籍ならではの利点もある. そこで本研究では, 電子書籍でありながら紙書籍に近い感覚が得られることを目標とした. 本システムは, ユーザーが歩き回れる仮想空間を構築すること, ユーザーの手の動きに対して仮想空間内で反応させること, 紙書籍に近い形状の電子書籍をモデリングレンダリングすること, の三つの機能に分けられる.Kinect を用いてユーザーの位置を, 基準マーカーを用いて顔の向きを検出し, 仮想空間内での移動を実現している. ユーザーの手の動きは Kinect で検出し, 仮想空間内での座標系と照らし合わせることで対応付け, ページ捲りや物の移動を実現している.Kinect で取得出来る関節のデータには時々大きなノイズが生じるため, ユーザーデータを元に補正方法を開発した. 読み進めた分のページを OpenGL を用いて三次元的に表現し, ページが捲れる様子も立体的にページを動かすことで, 読んでいる感覚を得やすい様に表現した. 本システムと Kindle for PC を用いて読後の記憶に関する比較実験を行った結果, 一つの課題の正答率は本システムが 27%,Kindle が 20% で若干優れている数字が出たが, もう一つの課題ではほぼ差が見られず, 有意な差とは言えなかった. 被験者の意見では, 本システムの方が読んでいる楽しさがあるという意見もあったが, 操作性や見た目などに対する課題も多かった.

2 27 MixedReality

3 MR MixedReality Kinect Kinect Kinect OpenGL Kindle for PC 27% Kindle 20%

4 Kinect v2[3] LeapMotion[4] VS MR MR Plug-in for Unity : MREAL[22] [25] VR Virtuix Omni[26] Variable Reality II[27] Kindle[28] MR

5 ( LeapMotion Kinect EC Kinect MR MR MR

6 1 1.1 CG Computer Graphics,Oculus Rift[12] GoogleGlass[2] HMD Head Mount Display),HMD,Kinect[3],LeapMotion[4] VR Virtual Reality HMD VRHMD,, PR,,, HMD [18] 1.2, MR, MR, 3

7 1.1: VR AR MR LeapMotion Kinect Kinect 4

8 8 5

9 2 VR AR MR 2.1 VR Virtual Reality AR Augmented Reality MR Mixed Reality ) VR ( 2.1) ( 2.2) AR CG ( 2.3, 2.4) MR AR VR AR MR PR 2.1: VR [5] 2.2: VR ( [6] ) 6

10 2.3: AR AR[7] 2.4: AR ( [8] ) 7

11 Kinect v2[3] Microsoft 3 Kinect v2( Kinect ( 2.5 Kinect. Xbox360 DanceEvolution Kinect Depth ) Depth Time of Flight(TOF:Kinect Depth Kinect 2.2 Kinect Color 2.6,Depth 2.7, 2.8 Color Depth Kinect LeapMotion[4] LeapMotion( 2.9), LeapMotion 150fps 120fps KeyTap (Circle Kinect,Oculus Rift[12] LeapMotion LeapMotion ( 2.10), ( 2.11) Oculus Rift HMD 8

2.1: Kinect v2 [9] OS Windows 8,Windows 8.1, Windows Embedded Standard 8, Windows Embedded Standard 8.1 36W CPU 2.66 GHz 64 (x64) USB 3.

12 2.1: Kinect v2 [9] OS Windows 8,Windows 8.1, Windows Embedded Standard 8, Windows Embedded Standard W CPU 2.66 GHz 64 (x64) USB 3.0, PCI-Express USB PCI-EX Gen2 (5Mbps) 2 GB RAM GPU DirectX 11 Visual Studio 2012 Visual Studio 2013.NET Framework : Kinect v2 [10] Color : , fps:30fps Depth( ) : ,fps:30fps, :0.5m 8.0m Player 6 Skeleton( 2 Joint 25 Hand State Angle : Kinect v2 9

13 2.6: Color 2.7: Depth 2.8: Kinect Body) 10

14 2.9: LeapMotion 2.10: Oculus Rift LeapMotion [11] Leap Motion VR Gallery 2.11: Oculus Rift LeapMotion [11] Planetarium 11

15 3 3.1,2013 [13] CD-ROM : : ): : : CD- ROM [14] Kindle ( ) iphone ipad [15][16] 50 [17] 3.2 VS Stavanger Anne Mangen, [19],, 12

16 [20][21],,,, 1,, 2010,, 13

17 4 4.1 MR MR Plug-in for Unity : MREAL[22] MR IT MREAL Unity D MREAL MR HM-A1 ( 4.1) MR ( 4.2) [22] MR MREAL : MREAL [24] ) 14

18 4.2: MREAL 2 [23] ) 4.3: MREAL [24] ) 15

19 16

4.4: AR LeapMotion ([25] ) 4.2 4.2.1 [25] CG AR MR AR CG Kinect(Kinect v1 LeapMotion CG Kinect LeapMotion LeapMotion 4.

20 4.4: AR LeapMotion ([25] ) [25] CG AR MR AR CG Kinect(Kinect v1 LeapMotion CG Kinect LeapMotion LeapMotion 4.4 LeapMotion LeapMotion 4.2.2,, 4.3 VR Virtuix Omni[26] VR, Virtuix Virtuix Omni 4.5 VR,,, 17

4.5: Virtuix Omni [26] PRODUCTS ) Omni, Omni Shoes Omni Tracking POD,Omni,Omni Omni, Omni Omni 4.6 4.3.

21 4.5: Virtuix Omni [26] PRODUCTS ) Omni, Omni Shoes Omni Tracking POD,Omni,Omni Omni, Omni Omni ,,, Variable Reality II[27] HYE SOO YANG Variable Reality II LeapMotion Oculus Rift AR VR

22 4.6: Virtuix Omni [26] ATX VR Meetup November 2015 ) 4.7: Variable Reality [27] Variable Reality Setup ) 4.8: AR [27] Variable Reality Augmented Book ) 19

23 4.9: AR [27] Variable Reality Demonstration Video ) 4.10: VR [27] Variable Reality Virtual Book ) 4.11: VR [27] Variable Reality Demonstration Video ) 20

24 4.12: Kindle for PC 4.13: Kindle for PC PDF AR ( ) VR ( ) VR AR Kindle[28] Amazon.com Kindle Amazon.com PC Windows8.1\8\7 Kindle for PC[29] Kindle for PC Enter 21

25 Kindle for PC PC PC 22

26 Kindle SonyReaderStore[30] PDF 1 23

27 5.1.3 MR 1 MR 24

28 MR MR MR CG 25

29 5.2 Kinect LeapMotion PC Web HMD Kinect LeapMotion Kinect PC Kinect LeapMotion Web USB HMD HDMI MR Kinect X Y HMD MR MR LeapMotion Kinect LeapMotion MR Kinect MR

30 5.2.4 Kinect 27

31 6 6.1 Kinect Windows8 Visual Studio( VS) C Kinect LeapMoiton HMD Web PC Kinect v2 SDK v LeapMotion SDK , Kinect for Windows v2,xbox One Kinect v2 Windows PC,PC HMD:HMZ-T3 HMD Sony 3D HMZ-T3 [31] PC HDMI 3D freeglut OpenGL OpenGL freeglut[32] freeglut GLUT the OpenGL Utility Toolkit), freeglut OpenCV OpenCV(Open Sourse Computer Vision Library)[33] GUI( 28

32 6.1: ARToolkit 6.2: ARToolkit OpenCV2.4.8 OpenCV OpenCV3.0 OpenCV OpenCV HP[33] OpenCV [34] VS VS Cmake(ver rc2) VS2015 VS2013 OpenCV :ARUco[35] OpenCV (opencv contrib) ARUco ARUco Web ARToolkit[36] OpenGL Kinect v2 OpenCV RGB Color ARUco ARToolkit hiro ( 6.1 ) ARUco ARUco 6.3 ID 29

33 6.3: AR [35] 30

34 6.4: : MR 6.4 ARUco PC PC HMD ARUco HMD MR,, 6.5 MR VS Web Kinect LeapMotion 31

35 6.6: Kinect MR ARUco ARUco Kinect 6.6 Kinect Color Depth PC Color X Depth Z Kinect Depth Kinect Kinect MR Kinect 2m 3, Kinect, 3, 6.8 Color, Depth, 32

36 6.7: Kinect 6.8: Color 6.9: 33

37 6.10: 34

38 6.11: ( Kinect SDK HMD ARUco 6.11 HMD ARUco Kinect ARUco,,, 4,,, Kinect Color ARUco,Color,ARUco Kinect Color, 3 1 HMD Kinect HMD 6.12 Kinect

39 6.12: ( [37] ) 36

40 6.13: 6.14: Kinect 37

41 6.15: LeapMotion HMD HMD LeapMotion ( 6.15) LeapMotion Kinect Kinect 4 ARUco Kinect MR X Y Z Kinect Kinect LeapMotion Kinect LeapMotion PC OpenGL, 38

42 6.16: LeapMotion MR LeapMotion SDK,MR 6.16 LeapMotion Web OpenCV Web Web Web LeapMotion,Web LeapMotion LeapMotion Web HMD OpenGL OpenGL OBB Oriented Bounding Box) OBB 39

43 6.17: 6.18: 40

44 6.19: 6.20: OBB 41

45 6.21: 6.22: OBB 6.20 OBB X Y Z 6.21 OBB OBB OBB OBB OBB 42

46 : MR 6.23 MR 43

47 7 LeapMotion Kinect Kinect Kinect 2 3 (JointType Head), (JoitType SpinShoulder) 3 (3Frame SpinShoulder) (3FrameBody) LeapMotion Kinect ( ) 27 ( ) 2015(EC2015) 9 26 HMD LeapMotion Kinect EC2015 Kinect MR LeapMotion MR 44

48 7.2.2 CG LeapMotion LeapMotion 25cm LeapMotion LeapMotion EC2015 LeapMotion LeapMotion 2 Leap- Motion LeapMotion LeapMotion 7.3 Kinect Kinect Web EC2015 LeapMotion Kinect MR 45

49 MR ( ) Kinect (JoitType Head) getdeptcoodinateaverage Z) get3framedepthcoodinatebodyaverage Kindle for PC[29] Kindle for PC HMD PC (a) HMD Web PC (b) Kinect (c) (d) 3 46

50 7.1: 7.2: 7.3: 47

51 7.1: [%] Kindle for PC (e) 1 2. Kindle for PC (a) 3 (b) , 3 6 4,Kinect Kindle for PC HMD - HMD, HMD, 48

52 -, PC,,, - MR,, -, -, ( ), 49

53 - - HMD Web HDMI USB,,, -,,,,,, -, - Kindle for PC Kindle, HMD HMD 50

54 ( 7.1) 51

55 8 8.1 Kinect,, MR,, MR Kinect, MR,Kinect Depth,, MR -, -,, 52

56 8.1.2 MR MR,OBB, LeapMotion Kinect MR Kinect LeapMotion Kinect LeapMotion Kinect MR MR Kinect, -,,MR -, x z, 53

58 [1] Oculus Rift Oculus VR, LLC [2] Google Glass Google [3] Xbox 360,Microsoft [4] LeapMotion [5] The London Heist Getaway Sony Morpheus VR E3 2015, [6] VR, [7] AR, [8] AR GooglePlay, kyorochanar [9] Xbox One Kinect Windows PC, productid [10] Kinect v1 Kinect v2, [11] Setup Leap Motion for VR Development, [12] Oculus, [13], 2013 ) 55

59 [14] 2014, \_.pdf [15] [16] Voxburner 62% of 16-24s prefer books as physical products prefer-books-as-physical-products [17] ( ) [18] 2014 ) [19] Want to Remember What You Read? Switch to Paper, what-you-read.html [20] 9 vs., [21] KLS2013JNd.pdf [22] MREAL IT 2013, [23] MR Plug-in for Unity IT [24] MREAL IT, [25] 2013) [26] Virtuix Omni,Virtuix, [27] Variable Reality II,HYE SOO YANG,

60 [28] Kindle, /b/ref=topnav_ storetab_kinc?ie=utf8&node= [29] Kindle for PC, [30] SonyReaderStore, [31] Personal 3D Viewer,Sony, [32] freeglut, [33] OpenCV, [34] OpenCV GitHub, [35] Automatic generation and detection of highly reliable fiducial markers under occlusion S. Garrido-Jurado, R. Munoz-Salinas, F.J Madrid-Cuevas, M.J. Mar n-jim enez, GarridoJurado2014.pdf [36] ARToolkit HLTL(Human Interface Technology), [37] UNITY-CHAN! OFFICIAL WEBSITE,

61 9,,,,,, 58

Raspberry Pi BF BF BF Raspberry Pi PC USB HDMI OS SD SD OS Raspberry Pi Model B MicroUSB MicroSD OS SD GPIO HDMI USB LAN Raspberry Pi MicroUSB MicroSD

Raspberry Pi BF BF BF Raspberry Pi PC USB HDMI OS SD SD OS Raspberry Pi Model B MicroUSB MicroSD OS SD GPIO HDMI USB LAN Raspberry Pi MicroUSB MicroSD Raspberry Pi Arduino Arduino Raspberry Pi Arduino URL Raspberry Pi HDMI USB SD https://www.raspberrypi.org/ Python, Scratch Arduino PC USB https://www.arduino.cc/ Arduino SDK Processing Kinect for Windows,