1. MIERUKEN 1 2 MIERUKEN MIERUKEN MIERUKEN: Speech Visualization System Based on Augmented Reality Yuichiro Nagano 1 and Takashi Yoshino 2 As the spread of the Augmented Reality(AR) technology and service, we are getting at sharing and visualizing various information on real environment. In this study, we focus on speech that used to transmission of information on daily life. We think that speech visualization can support various situations in daily activities. We have developed speech visualization system MIERUKEN based on AR. In this paper, we present the result of a trial experiment and discuss the evaluation of three methods for visualized speech. 1 Graduate School of Systems Engineering, Wakayama University 2 Faculty of Systems Engineering, Wakayama University ARToolKit PTAM Augmented RealityAR 1),2) 1 ARis 2 Web GPSHMD AR AR AR 3) AR AR MIERUKEN MIERUKEN 3 AR 2. 2.1 4),5) Levin 1 : http://sekaicamera.com/ 2 ARis: http://www.geishatokyo.com/jp/ar-figure/ 1 c 2009 Information Processing Society of Japan
4) Lewis 5) 6) 2.2 7),8) 7) IC Web 8) 3 MySpace: http://www.myspace.com/ 4 mixi: http://mixi.jp/ MySpace 3 mixi 4 SNS SNS 9),10) SNS mixi 9) SNS 11) 3. MIERUKEN 3.1 MIERUKEN AR ( 1 ) 2 c 2009 Information Processing Society of Japan
( 2 ) 3.2 1 MIERUKEN MIERUKEN AR Julius 12) GPS AR AR AR HMD AR ARToolKit 1) OpenGL C# C++ 2 AR HMD HMD Viuzix iwarevr920 3 Web Microsoft LifeCam Show 30 800x600px GPS AR Fig. 1 1 The configuration of MIERUKEN. 2 AR Fig. 2 The configuration of AR user hardware. 3 c 2009 Information Processing Society of Japan
3.3 MIERUKEN ( 1 ) 3 MIERUKEN 3 3 3 Web Mecab 13) Web Bing API 5 ( 2 ) HMD AR 4 HMD 3 4. 4.1 2 Fig. 3 Fig. 4 a. b. 3 3 The three kinds of visualization method. 4 The function of look back to a past speech log. 14) 5 Bing API: http://www.bing.com/developers 4 c 2009 Information Processing Society of Japan
() () 2 2 2 3 3 GPS ARToolKit 5 2 6 15),16) 13 3 4.2 1 case1 case2 case1 5, F: M: F: M: F: M:.... 5 Fig. 5 Example of speech texts and a question. 1 Table 1 The experiment patterns. Text1 Text2 Text3 Text4 Text5 Text6 ( ) ( ) () ( ) ( ) () case1 case2 5 5. 2 Q1 100% 90% 40% 80% 5 c 2009 Information Processing Society of Japan
Table 2 2 The result of a content understanding test. Table 3 3 5 The result of a questionnaire survey using a five-point Likert item. Q1 Q2 Q1 Q2 Q1 Q2 Q1 Q2 Q1 Q2 Q1 Q2 user01 0.83 0.94 0.80 0.91 0.97 0.17 user02 0.73 0.86 0.79 0.90 1.00 0.57 user03 0.76 0.90 0.92 0.93 0.94 0.20 user04 0.89 0.97 0.96 1.00 0.88 0.11 user05 0.83 0.92 0.86 0.83 0.94 0.17 user06 0.89 0.88 0.96 0.83 0.92 0.67 user07 1.00 0.88 0.75 0.81 0.83 0.42 user08 0.80 0.73 0.73 0.86 0.89 0.40 user09 0.96 0.92 0.83 0.83 0.83 0.40 user10 1.00 0.88 0.83 0.94 1.00 0.13 0.4 0.87 1.0 0.89 0.9 0.84 0.8 0.89 0.3 0.92 0.2 0.32 0.52 0.30 0.00 0.27 0.32 0.34 0.42 0.24 0.48 0.24 0.42 0.47 Q1: (:, : ) Q2: 3 0: 0.5: 1: 3 30% 20% Q2 3 0: 0.5:1: 3 80% 80% 30% 3 5 (1) (2) (3) (4) (5) (6) (7) (3) (4) 4.2 0.79 4.8 0.42 (1) 4.0 0.82 3.7 0.82 1.9 0.74 1.9 0.99 3.4 0.84 4.0 0.82 (2) 3.6 0.84 3.7 1.16 2.4 0.97 2.3 1.57 1.9 0.74 3.8 0.92 (3) 1.6 0.84 3.3 0.95 1.7 1.06 2.7 1.06 (4) 2.2 0.92 3.4 0.70 (5) 3.6 0.70 4.2 0.92 (6) () 4.4 0.70 (7) 3.0 0.94 1234 5 5 6. 6.1 3 Q1 80% 40%40 3 6 c 2009 Information Processing Society of Japan
100% 30%70 3 90% 20% 70 3 90% 3 2 2 2 6.2 HMD 45 24% 11 60(10 6) 15% 9 3 (5) 3.6 4.2 ( ) 3 (6) 4.4 3 (7) 3.0 7. AR MIERUKEN MIERUKEN ( 1 ) 3 7 c 2009 Information Processing Society of Japan
( 2 ) (B)(19300036) 1) Kato, H., Billinghurst, M.: Marker Tracking and HMD Calibration for a Videobased Augmented Reality Conferencing System, Proc. IEEE and ACM International Workshop on Augmented Reality (IWAR 99), pp.85 (1999). 2) Klein, G., Murray, D.: Parallel Tracking and Mapping for Small AR Workspaces, Proc. IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 07), pp.1-10 (2007). 3) Stafford, A., Piekarski, W. and Thomas, B.: Implementation of God-like Interaction Techniques for Supporting Collaboration Between Outdoor AR and Indoor Tabletop Users, Proc. IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 06), pp.165-172 (2006). 4) Levin, G., Lieberman, Z.: In Situ Speech Visualization in Real-Time Interactive Installation and Performance, Proc. International symposium on Non-photorealistic animation and rendering (NPAR 04), pp.7-14 (2004). 5) Lewis, J., Assogba, Y.: Taking sides: dynamic text and hip-hop performance, Proc. ACM international conference on Multimedia (MULTIMEDIA 06), pp.744-747 (2006). 6), :,, 2009-HCI-132, pp.1-8 (2009). 7), :,, 2002-GN-31, pp.109-114 (2002). 8),, :,, Vol.89, No.3, pp.206-212 (2006). 9) : SNS DI, (DICOMO 07), pp.1510-1513 (2007). 10),, :, 67, 3, pp.157-158 (2005). 11) Fish, R.S., Kraut, R.E. and Chalfonte, B.L.: The VideoWindow System in Informal Communications, Proc. ACM conference on Computer-supported cooperative work (CSCW 90), pp.1-11 (1990). 12), : Julius,, Vol.20, No.1, pp.14-19 (2005). 13) Kudo, T., Yamamoto, K. and Matsumoto, Y.: Applying Conditional Random Fields to Japanese Morphological Analysis, Proc. Conference on Empirical Methods on Natural Language Processing (EMNLP 04), pp.230-237 (2004). 14) :,, 2, pp.41-49 (1985). 15),, :, (2005). 16),, :, (2005). 8 c 2009 Information Processing Society of Japan