: VR
1 VR Unity VR Leap Motion 4 0 Leap Motion Kinect VR
2 1 7 1.1............................................ 7 1.1.1............................... 7 1.1.2...................................... 7 1.1.3....................... 7 1.2............................................ 7 1.3....................................... 8 2 10 2.1 Hand gesture-based interactive puppetry system to assist storytelling for children... 10 2.2 Interaction with virtual objects in a natural way..................... 10 2.3 Multi-sensor Exercise-based Interactive Games for Fall Prevention and Rehabilitation. 10 2.4 Using virtual reality to analyze sports performance................... 10 2.5...................... 10 2.6................ 11 2.7........................................ 11 3 12 3.1............................................ 12 3.1.1..................................... 12 3.1.2.................................. 12 3.1.3.................................... 13 3.2........................................... 14 3.3............................................ 14 3.3.1.................................... 15 4 18 4.1 Leap Motion........................................ 18 4.2 Unity........................................... 19 4.2.1............................... 19 4.2.2..................................... 19 4.2.3 Unity................................ 19 4.2.4..................................... 21
3 5 22 5.1........................................... 22 5.2............................................. 23 5.3 Unity Leap Motion................................ 23 5.4........................................... 24 5.5.............................. 24 5.5.1................................ 25 5.6........................................ 25 5.6.1........................ 25 5.7......................................... 26 5.7.1........................ 28 5.8......................................... 28 5.9......................................... 30 6 32 6.1............................................ 32 6.1.1 1................................... 32 6.1.2 2................................... 33 6.2............................................ 34 6.2.1 1............................... 34 6.2.2 2............................... 34 7 36 8 37 8.1............................... 37 8.2................................... 37 8.3............................... 37
4 6.1 1............................... 34 6.2 1............................... 35 6.3 2..................................... 35
5 1.1.................................... 8 3.1 1.................................. 12 3.2 1.................................. 12 3.3 2.................................. 13 3.4 2.................................. 13 3.5 3.................................. 13 3.6 3.................................. 13 3.7........................................... 13 3.8......................................... 13 3.9......................................... 14 3.10......................................... 14 3.11.......................... 16 3.12...................... 17 3.13..................... 17 4.1 Leap Motion........................................ 18 4.2................. 19 4.3..................... 20 5.1............................... 22 5.2................................... 23 5.3 Leap Motion............................. 24 5.4..................................... 29
6 Listings 4.1 C# UnityScript............................ 20 5.1................................... 24 5.2.................................. 25 5.3.............................. 26 5.4.............................. 27 5.5......................................... 29 5.6............................................. 30
7 1 1.1 1.1.1 2016 VR VR VR 1.1.2 1.1.3 [9] 1.2 1 VR VR 1
1 8 1.1: 1.3 ( 1 ) 2 3 4 5
1 9 6 7 8
10 2 2.1 Hand gesture-based interactive puppetry system to assist storytelling for children [3] Leap Motion Unity Serious Game 2.2 Interaction with virtual objects in a natural way Prazina [5] Leap Motion 3D 2.3 Multi-sensor Exercise-based Interactive Games for Fall Prevention and Rehabilitation Santos [6] Leap Motion Kinect 1 VR 10 2.4 Using virtual reality to analyze sports performance Bideau [1] 2.5 [13] 1 http://www.xbox.com/ja-jp/kinect/
2 11 2.6 [12] 2.7 [8] [10]
12 3 3.1 [11] 3.1.1 1 3.1: 1 3.2: 1 3.1.2 2 1 http://www.nihonkiin.or.jp/teach/gogame/index.html
3 13 3.3: 2 3.4: 2 3.5: 3 3.6: 3 3.7: 3.8: 3.1.3 2 3.9 3.10 3.9
3 14 3.9: 3.10: 3.2 3.3 1 150 300 50 60 1 1 3 1 5 19 19 = 361 9 9 = 81
3 15 3.3.1 3.11 Leap Motion Leap Motion Leap Motion Unity 3.12 Unity 3.13
3 16 3.11:
第 3 章 提案システム 17 図 3.12: ゲーム上で碁石を擬似的に掴んでいる様子 図 3.13: 壁と天井を表示した時のスクリーンショット
18 第4章 実装環境 実装には Unity Technorogies 社のゲームエンジンである Unity [2] を用いた また機器として Leap Motion 社の VR 機器である Leap Motion [4] を用いた 使用したコンピュータは MacBook Pro (Retina, 15-inch, Late 2013) である 4.1 Leap Motion 図 4.1: Leap Motion Leap Motion は赤外線 LED と 2 個の赤外線カメラを用いたハンドトラッキング 3D 入力デバイ スの一つで 高精度モードで 80fps 速度重視モードで 290fps ほど (USB3.0 接続時) のトラッキ ング速度を持つ 両手十指に反応することができ 最高精度で 10µm 程度の移動距離を取得する ことができる [7]
4 19 SDK Windows Orion Mac Linux Version2 C Orion C++ C# Objective-C Unity Java Python Javascript Unreal Engine 4 Leap Motion SDK version2.3.1 Unity Asset Unity 4.2 Unity Unity 3D 5.3.5f1 4.2.1 Unity 4.2.2 Unity Unity Unity Unity 4.2: 4.2.3 Unity Unity Javascript C# Boo 1 3 1 Python
第 4 章 実装環境 20 図 4.3: アセットストアの様子 モザイク処理済み 1 2 using UnityEngine; using System.Collections; 3 4 public class UnityScript : MonoBehaviour { 5 // Use this for initialization void Start () { 6 7 8 } 9 10 // Update is called once per frame void Update () { 11 12 13 } 14 15 } Listing 4.1: C#による基本になる UnityScript listing 4.1 は基本的な Unity におけるスクリプト (C#) である スクリプトを生成すると Update(){} と Start(){}の 2 つのメソッドが初めから記述されている Update() このスクリプトが適用されているゲームオブジェクトが毎フレーム呼び出されるメソッド である Start() このスクリプトが適用されているゲームオブジェクトが初めて Update(){}を呼ぶ前に一度 だけ呼び出されるメソッドである これらはイベントメソッドと呼ばれ これら 2 つ以外にも多くのメソッドが存在する
4 21 4.2.4 Unity
22 5 Unity 5.1 5.1: 3D Unity
5 23 5.2 5.2: Unity Directional Light 5.2 5.3 Unity Leap Motion 5.3 Unity x = 10, y = 10, z = 10
5 24 5.3: Leap Motion 5.4 The Texture Lab Grass 01 Game-Ready 1 Rispat Momit Simple Cloud System 2 5.5 1 void OnTriggerEnter(Collider other){ 2 Destroy(other.gameObject); 3 } Listing 5.1: DelObject.cs 1 https://www.assetstore.unity3d.com/jp/#!/content/28929 2 https://www.assetstore.unity3d.com/jp/#!/content/6715
5 25 5.5.1 OnTriggerEnter listing5.1 OnTriggerEnter(Collider other){} listing4.1 Update()Start() Destroy Destroy(other.gameObject) 5.6 1 void Start () { 2 3 //generate box 4 GameObject parentobject = GameObject.FindGameObjectWithTag("Line"); 5 Vector3 linepos = new Vector3(5f,1.51f, 1f); 6 7 Quaternion q = new Quaternion(); 8 q = Quaternion.identity; 9 10 for(int i=0;i<9;i++){ 11 Instantiate(linePrefab,linePos,q); 12 linepos.z = 1f; 13 } 14 15 q = Quaternion.Euler(0,90,0); 16 linepos = new Vector3(1f,1.51f, 5f); 17 18 for(int i=0;i<9;i++){ 19 Instantiate(linePrefab,linePos,q); 20 linepos.x += 1f; 21 } 22 23 } Listing 5.2: 8(m) 0.01 0.02 5.6.1 Quaternion Quaternion 4 R 4 e
5 26 1 e = i j k (5.1) H q = a + bi + c j + dk (5.2) i, j, k i 2 = j 2 = k 2 = 1 (5.3) i j = k, jk = i, ki = j (5.4) ji = k, k j = i, ik = j (5.5) q = a bi c j dk (5.6) (α, β, γ) v θ q = cos θ 2 + α sin θ 2 i + β sin θ 2 j + γ sin θ 2 k v = qvq Quaternion.identify Static Instantiate Instantiate(Gameobject object,vector3 v,quaternion q){} 3 5.7 1 public GameObject stoneblack; 2 public GameObject stonewhite; 3
5 27 4 bool black; 5 6 Quaternion q = new Quaternion(); 7 8 // Use this for initialization 9 void Start () { 10 q = Quaternion.identity; 11 black = true; 12 } 13 14 // Update is called once per frame 15 void Update () { 16 if(input.getkeydown(keycode.h)){ 17 18 try{ 19 GameObject stone = 20 GameObject.Instantiate(black?stoneBlack:stoneWhite, 21 this.transform.position, q)as GameObject; 22 23 black = black?false:true; 24 } catch (System.NullReferenceException e){ 25 Debug.Log("Cannot generate a stone."); 26 } 27 } 28 } Listing 5.3: 1 2 public GameObject stonewhite; 3 public GameObject stoneblack; 4 5 public GameObject[] FingerObjects; 6 7 HandController handcontroller; 8 Finger finger; 9 10 11 Quaternion q = new Quaternion(); 12 GameObject hand; 13 14 bool black = true; 15 16 const int SPEED = 100; 17 18 // Use this for initialization 19 void Start () { 20 21 q = Quaternion.identity; 22 23 } 24 25 26 // Update is called once per frame 27 //generate stone
5 28 28 void Update () { 29 30 hand = GameObject.Find( 31 "/SaltLightFullRightHand(Clone)/HandContainer"); 32 33 if(hand == null) 34 hand = GameObject.Find( 35 "/SaltLightFullLeftHand(Clone)/HandContainer"); 36 37 if(input.getkeydown(keycode.z) 38 Input.GetKeyDown(KeyCode.Space)){ 39 40 try{ 41 GameObject stone = GameObject.Instantiate( 42 black?stoneblack:stonewhite, 43 hand.transform.position,q) as GameObject; 44 45 black = black?false:true; 46 47 } catch (System.NullReferenceException e){ 48 Debug.Log("Cannot generate a stone."); 49 } 50 } 51 52 } Listing 5.4: listing 5.3 listing 5.4 listing 5.3 H listing 5.4 5.7.1 GameObject.Find(String pass) listing 5.4 GameObject.Find(String pass){} String pass GameObject 5.8
5 29 5.4: 1 void Update () { 2 3 if(input.getkeydown(keycode.w)){ 4 5 if(wallmode){ 6 7 Debug.Log("Dell Wall"); 8 9 foreach(gameobject delwall 10 in GameObject.FindGameObjectsWithTag("Wall")) 11 Destroy(delWall); 12 13 wallmode = false; 14 15 }else{ 16 17 Debug.Log("Gen wall"); 18 Instantiate(wall, 19 new Vector3(0,5, 5), 20 Quaternion.identity); 21 22 Instantiate(wall, 23 new Vector3(10,5, 5), 24 Quaternion.identity); 25 26 Instantiate(wall, 27 new Vector3(5,5,0), 28 Quaternion.Euler(0,90,0)); 29 30 Instantiate(wall, 31 new Vector3(5,10, 5), 32 Quaternion.Euler(0,0,90)); 33 34 wallmode = true;
5 30 35 } 36 } 37 38 } Listing 5.5: 5.9 1(m) 9 9 0.8 0.78 0.04 0 µ 0.3 e 0.6 1 public Rigidbody rb; 2 3 Vector3 roundpos; 4 5 bool ongoban; 6 7 public bool destroymode; 8 9 static int LIMIT_SPEED = 2; 10 11 bool frozen; 12 // Use this for initialization 13 void Start () { 14 rb = GetComponent<Rigidbody>(); 15 rb.constraints = RigidbodyConstraints.None; 16 ongoban = false; 17 destroymode = true; 18 frozen = false; 19 // destroymode = false; 20 Debug.Log("destroyMode:" + destroymode); 21 22 } 23 24 // Update is called once per frame 25 void Update () { 26 27 } 28 void OnTriggerEnter(Collider other){ 29 30 if (other.gameobject.comparetag("grid")) { 31 Debug.Log("StonePos" + other.gameobject.transform.position); 32 } 33
5 31 34 if(destroymode){ 35 Debug.Log("Collision! on " + 36 (other.gameobject.comparetag("stoneblack")?"stoneblack":"") 37 + (other.gameobject.comparetag("stonewhite")?"stonewhite":"") ); 38 if((other.gameobject.comparetag("stonewhite")) 39 other.gameobject.comparetag("stoneblack")) { 40 GameObject.Destroy(other.gameObject); 41 } 42 } 43 44 45 } 46 47 void OnCollisionStay(Collision other){ 48 if (!frozen & other.gameobject.comparetag("grid")){ 49 if (rb.velocity.sqrmagnitude == 0 ){ 50 roundpos = new Vector3(Mathf.RoundToInt(rb.position.x), 51 rb.position.y, 52 Mathf.RoundToInt(rb.position.z)); 53 this.transform.position = roundpos; 54 rb.constraints = RigidbodyConstraints.FreezeAll; 55 frozen = true; 56 } 57 if (!ongoban && rb.velocity.sqrmagnitude < LIMIT_SPEED) { 58 //Stop Stones 59 ongoban = true; 60 this.transform.rotation = Quaternion.identity; 61 rb.constraints = RigidbodyConstraints.FreezePositionX 62 RigidbodyConstraints.FreezePositionZ 63 RigidbodyConstraints.FreezeRotationX 64 RigidbodyConstraints.FreezeRotationZ ; 65 } 66 67 } 68 } Listing 5.6:
32 6 6.1 6.1.1 1 2 25 1 1 2 Leap Motion 10 1
6 33 2 1 10 70cm 8 6.1.2 2 1 22 23 2 2 e 1 Leap Motion 10 1 5 2 1 10 70cm 5 3
6 34 4 e 0.6 0 0.99 6.2 6.2.1 1 1 2 6.2.2 2 1 1 6.1: 1 1 2 3 4 5 1 7 6 7 6 5 2.8 1 2.8 1.4 0 2 2
6 35 6.2: 1 1 2 3 4 5 2 9 8 4 7 2 3.9 3 3.16 2 3 6.3: 2 1 2 3 4 5 1 11cm 9cm 10cm 2cm 4cm 2 10cm 15cm 17cm 7cm 3cm 1 4 2
36 7 2 1,2 0
37 8 Leap Motion Unity VR VR VR 8.1 8.2 Leap Motion Kinect VR Oculus Rift Gear VR, HTC Vive (HMD) 8.3
38. 2 1 4 OB.
39 [1] Benoit Bideau, Richard Kulpa, Nicolas Vignais, Sébastien Brault, Franck Multon, and Cathy Craig. Using virtual reality to analyze sports performance. IEEE Computer Graphics and Applications, Vol. 30, No. 2, pp. 14 21, 2010. [2] Unity Tecnorogies Japan. Unity - game engine, tools and multiplatform. https://unity3d.com/jp/unity/index.html, 2016-12-01. [3] Hui Liang, Jian Chang, Ismail K Kazmi, Jian J Zhang, and Peifeng Jiao. Hand gesture-based interactive puppetry system to assist storytelling for children. The Visual Computer, pp. 1 15, 2016. [4] Leap Motion. Leap motion mac & pc motion controller for games, design, virtual reality & more. https://www.leapmotion.com/, 2016-12-01. [5] I Prazina, K Balic, K Prses, S Rizvic, and V Okanovic. Interaction with virtual objects in a natural way. In MIPRO 2016-39th International Convention, pp. 376 379, 2016. [6] António Santos, Vânia Guimarães, Nuno Matos, João Cevada, Carlos Ferreira, and Inês Sousa. Multi-sensor exercise-based interactive games for fall prevention and rehabilitation. In Proceedings of the 9th International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 15, pp. 65 71. ICST, 2015. [7] Frank Weichert, Daniel Bachmann, Bartholomäus Rudak, and Denis Fisseler. Analysis of the accuracy and robustness of the leap motion controller. Sensors, Vol. 13, No. 5, pp. 6380 6393, 2013. [8],,.. (GI), Vol. 2007, No. 20 (2007-GI-017), pp. 49 55, 2007. [9]. 2.., Vol. 60, No. 4, pp. 491 494, 2006. [10],.. GI, Vol. 2000, No. 27, pp. 87 92, mar 2000. [11].. http://www.nihonkiin.or.jp/match/kiyaku/zenbun.html, 2016-01-03. [12],,.. 21, pp. 77 77., 2005.
8 40 [13],,.. (GI), Vol. 2014, No. 2, pp. 1 5, 2014.