Vol.6 No (Sep. 2013) [2] LDA Latent Dirichlet Allocation [3] [4] [2] [5] [6], [7] [8], [9] [10] [11] LDA [4] LDA [2], [4] CGM [12], [13], [14]

Transcription

1 Vol.6 No (Sep. 2013) ,a) , CGM 2 CGM Research Concerning Evaluation Indexes of Topics Based on Important Degree of Focused Information Shigenori Tanaka 1 Kenji Nakamura 2 Yuhei Yamamoto 3 Naoaki Yanagida 3,a) Received: March 20, 2013, Accepted: July 5, 2013 Abstract: With the spread of CGM, all kinds of reactions to a topic such as questions, opinions, impressions, and provision of information started to be posted on the Internet. Since information contained in the contents of those posts is based on diverse sense of values, a method for extracting only the useful from the posts is needed. Existing methods suggest approaches to evaluate the degree of drawing attention based on the evaluation index of burst, or the posted information according to the amount of information that is an index to evaluate importance. However, burstiness does not help evaluating the importance of the contents of a post, because burstiness is index based on the number of the post. And amount of information does not help evaluating the degree to which it draws users attention, because amount of information is index based on the contents of a post. There are problems that burstiness does not help evaluating the importance of the contents of a post, and that the amount of information does not help evaluating diverse reactions of users to the post. This study proposes a new index for evaluating a topic according to the degree to which it draws users attention and the importance of the contents of a post by combining these two indices. And we demonstrate the effectiveness of the proposed index by the demonstration experiments. Keywords: evaluation indexes of information, amount of topic information, burst, CGM, data mining 1 Faculty of Informatics, Kansai University, Takatsuki, Osaka , Japan 2 Faculty of Information Technology and Social Science, Osaka University of Economics, Osaka , Japan 3 Graduate School of Informatics, Kansai University, Takatsuki, Osaka , Japan a) k086121@gmail.com 1. SNS Social Network Service CGM Consumer Generated Media [1] c 2013 Information Processing Society of Japan 69

2 Vol.6 No (Sep. 2013) [2] LDA Latent Dirichlet Allocation [3] [4] [2] [5] [6], [7] [8], [9] [10] [11] LDA [4] LDA [2], [4] CGM [12], [13], [14], [15], [16] [12], [13], [14], [15], [16] [2] CGM Fig. 1 Relationship between burst and amount of information. [16] [17] [18] CGM [17] Facebook c 2013 Information Processing Society of Japan 70

3 Vol.6 No (Sep. 2013) STEP 2.3 DB STEP Fig. 2 Flow of process. 3.1 [16] 3 DB DB DB 3 DB DB STEP 1 STEP 1 STEP 2 STEP 1 STEP 2.1 STEP 2.3 STEP 2.1 DB DB DB DB STEP 2.2 DB DB STEP 2.1 DB 3.2 [16] [16] t x Burst(t, x) t B(t) ={Burst(t, 1),Burst(t, 2),,Busrt(t, x)} 3.3 [17] (1) N x = {w 1,w 2,,w k,,w N } t H(t, x) (1) N H(t, x) = P twk log 2 P twk (1) k=1 P twk t w k P twk P (1) P twk c 2013 Information Processing Society of Japan 71

4 Vol.6 No (Sep. 2013) P twk (2) 1 totalappear(t) (w k is new word) P twk = (2) 1 (w k is not new word) (2) totalappear(t) t (2) 0 Wsize Wsize totalappear(t) w k Wsize w k 0 P twk 1 w k P twk log 2 P twk 0 H User (t, x) H News (t, x) t x H Topic (t, x) (3) H Topic (t, x) =H User (t, x)+h News (t, x) (3) t H(t) ={H Topic (t, 1),H Topic (t, 2),, H Topic (t, x)} Burst(t, x) (4) Burst (t, x) = Burst(t, x) min(b(t)) max(b(t)) min(b(t)) (4) H Topic (t, x) Burst (t, x) H Topic (t, x) t x D Focused (t, x) (5) D Focused (t, x) =Burst (t, x) H Topic(t, x) (5) 3.5 Burst (t, x) H Topic (t, x) 1 Burst (t, x) H Topic (t, x) t x D Unfocused (t, x) (6) D Unfocused (t, x) =(1 Burst (t, x)) H Topic(t, x)(6) 3.6 Stopper Stopper t DF(t) = {D Focused (t, 1),D Focused (t, 2),,D Focused (t, x)} Stopper (7) Stopper(DF(t)) = max(df(t)) α (7) (7) α 0 α 1 Stopper c 2013 Information Processing Society of Japan 72

5 Vol.6 No (Sep. 2013) 1 Table 1 Experiment environment. OS Windows7 Professional 32 bit Visual C# Intel R Core TM i Processor 8GB 1 1 [17] [16] LDA [4] N β Wmin Amin Cmin Wmax Wsize LDA k T 1 T 2 J N β Wmin Amin Cmin Wmax N β Wmin Amin Cmin Wmax 6 [16] N =50 β =0.4 Wmin =1 Amin =15 Cmin =15 Wmax = Wsize Wsize Wsize = LDA k T 1 T 2 J LDA k T 1 T 2 J 4 [4] k =30 T 1 =7 T 2 =14 J = , , MeCab [20] IPA STEP 1 IPA STEP 2 1 STEP 3 STEP 2 1 c 2013 Information Processing Society of Japan 73

6 Vol.6 No (Sep. 2013) STEP 1 5,000 STEP 2 IPA *1 STEP 3 IPA STEP STEP 5 STEP 5.1 STEP 5.5 STEP 5.1 STEP 5.2 N News 10 2,250 STEP 5.3 α News STEP 5.4 N News α News STEP 5.5 N News (1 α News ) *1 IPA [19] 0 STEP 1 IPA STEP 2 IPA STEP 3 0 C MaxUserT [19] β 0 C MaxUserF STEP 4 STEP 4.1 STEP 4.4 STEP 4.1 N User STEP 4.2 α User STEP 4.3 N User α User STEP 4.4 N User (1 α User ) c 2013 Information Processing Society of Japan 74

7 Vol.6 No (Sep. 2013) 2 6 STEP 1 50 STEP 2 STEP 2.1 STEP 2.3 STEP 2.1 STEP 1 50 STEP STEP STEP t STEP t t m t Cover(t, m) Cover(t, m) (8) Cover(t, m) = totalappear(t, m) 6 k=1 totalappear(t, k) (8) (8) totalappear(t, m) m t Cover(t, m) STEP STEP STEP 2.2 STEP 1 50 STEP STEP 2.3 STEP 2.2 STEP STEP STEP 2 Fig. 3 Results by STEP 2. 2 Table 2 Cumulative coverage of integrated news articles STEP 3 3 STEP 3.1 STEP 3.2 STEP STEP STEP STEP t STEP STEP (8) (8) m STEP c 2013 Information Processing Society of Japan 75

8 Vol.6 No (Sep. 2013) STEP 1 50 Google STEP STEP ch.net FC2 bbs.fc2.com Yahoo! chiebukuro.yahoo.co.jp 3 Table 3 Domains for crawling news articles. sankei.jp.msn.com nikkei.com mainichi.jp onayamifree.com Yahoo! Table 4 Domains for crawling users posts. 2ch.net 272 musyoku.com 50 web2ch.org 260 e-mansion.co.jp 47 groups.google.com 254 machi.to 40 desktop2ch.net 234 onayamifree.com 32 chiebukuro.yahoo.co.jp 172 ezbbs.net 31 jbbs.livedoor.jp 141 bbs.fc2.com 25 qa.itmedia.co.jp 75 2chan.net 25 shizu.0000.jp 73 progoo.com 23 bakusai.com 59 community.teacup.com 11 mikle.jp 50 meiwasuisan.com 11 5 Table 5 Topics using by experiments. ID ID F Facebook B-1 6 CATS 18 7 Surface c 2013 Information Processing Society of Japan 76

9 Vol.6 No (Sep. 2013) 6 Table 6 Examples of threads removed and adopted by Akutagawa award. bbs.fc2.com bbs.fc2.com ch-sakura.jp machi.to shizu.0000.jp 2ch.net 2ch.net 2ch.net desktop2ch.net ezbbs.net Part STEP 1 4 STEP 2 Google 40 STEP [17] [16] LDA [4] LDA 4 1 Fig. 4 Evaluation indexes for Experiment Table 7 Parameters for creating artificial data in Experiment 1. α News 0.30 α User 0.30 β 3 C MaxUserT 100 C MaxUserF 30 4 LDA LDA (7) (7) α α F α STEP LDA C MaxUserT 100 C MaxUserF =10 4 =30 5 =30 6 =15 7 =22 C MaxUserF =30 c 2013 Information Processing Society of Japan 77

10 Vol.6 No (Sep. 2013) 5 1 Fig. 5 Artificial data in Experiment STEP 2 α STEP 2.1 STEP 2.2 STEP 2.1 STEP 2.2 STEP 1 F STEP LDA 6 6 α F F α F α F 0.70 α = α =0.39LDA 0.70 α = α =0.10α Fig. 6 Extraction accuracy of correct data. 8 F Table 8 Maximum value of F-measure. α (/) (/) F (24/40) 0.75(24/32) (24/37) 0.75(24/32) 0.70 LDA (19/22) 0.59(19/32) (23/28) 0.72(23/32) Table 9 Average analysis time. 00: : LDA 03: : F t t t(200) = 2.11 c 2013 Information Processing Society of Japan 78

11 Vol.6 No (Sep. 2013) Fig. 7 7 Surface Analysis result about Microsoft and Surface. p<.05 LDA Welch t t( ) = 4.77 p <.01 LDA t t(200) = 0.31,n.s. α α α = t(70) = 2.07 p <.05 2 F α =0.10 α = LDA LDA 0.15 LDA 2 F CGM LDA CGM STEP STEP 2 STEP 3 STEP 4 c 2013 Information Processing Society of Japan 79

12 Vol.6 No (Sep. 2013) ID Surface ID7 Surface Surface Table 10 Data extracted by Microsoft and Surface. MicrosoftSurface 2012/06/19 nikkei.co.jp Microsoft Surface 2012/06/24 2ch.net Microsoft Surface 2012/06/26 2ch.net Surface /08/18 2ch.net 11 Table 11 Examples of extracted noise. desktop2ch.net desktop2ch.net w desktop2ch.net musyoku.com musyoku.com ID B-1 ID17B Surface Table 12 Classification of users posts concerning Microsoft and Surface UNIX XBOX c 2013 Information Processing Society of Japan 80

13 Vol.6 No (Sep. 2013) B Surface 13 B-1 Table 13 Data extracted by Grand prix of B-1. B /11/12 nikkei.co.jp B /11/13 2ch.net B /11/14 2ch.net Fig. 8 8 B-1 Analysis result about Grand prix of B-1. Table B-1 Classification of users posts concerning Grand prix of B c 2013 Information Processing Society of Japan 81

14 Vol.6 No (Sep. 2013) Fig. 9 9 Analysis result about Kansai university and Wrestling club. 15 Table 15 Data extracted by Kansai university and Wrestling club. 2011/04/ /04/ /04/12 [] chiebukuro.yahoo.co.jp [] chiebukuro.yahoo.co.jp mainichi.jp LDA c 2013 Information Processing Society of Japan 82

15 Vol.6 No (Sep. 2013) N β α Wmin Amin Cmin Wmax Wsize 3 3 [1] content/ pdf [2] Kleinberg, J.: Bursty and Hierarchical Structure in Streams, Proc. 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp , ACM (2002). [3] Blei, D., Ng, A. and Jordan, M.: Latent Dirichlet Allocation, Journal of Machine Learning Research, Vol.3, pp , JMLR (2003). [4] t-lda Vol.23, No.10, pp.1 6, (2010). [5] Kumar, R., Novak, J., Raghavan, P. and Tomkins, A.: On the Bursty Evolution of Blogspace, Proc. 12th International Conference on World Wide Web, pp , ACM (2003). [6] Platakis, M., Kotsakos, D. and Gunopulos, D.: Discovering Hot Topics in the Blogsphere, Proc. 2nd Panhellenic Scientific Student Conference on Informations, pp , Related Technologies and Applications EUREKA 2008 (2008). [7] Vol.106, No.38, pp.51 56, (2006). [8] He, Q., Chang, K. and Lim, E.: Using Burstiness to Improve Clustering of Topics in News Streams, Proc th IEEE International Conference on Data Mining, pp , IEEE (2007). [9] He, Q., Chang, K., Lim, E. and Zhang, J.: Bursty Feature Representation for Clustering Text Streams, Proc. 7th SIAM International Conference on Data Mining, pp , SIAM (2007). [10] Lappas, T., Arai, B., Platakis, M., Kotsakos, D. and Gunopulos, D.: On Burstiness-Aware Search for Document Sequences, Proc. 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp , ACM (2009). [11] Sakkopoulos, E., Antoniou, D., Adamopoulou, P., Tsirakis, N. and Tsakalidis, K.: A Web Personalizing Technique Using Adaptive Data Structures: The Case of Bursts on Web Visits, Journal of Systems and Software, Vol.83, pp , Elsevier (2010). [12] Zhu, Y. and Shasha, D.: Efficient Elastic Burst Detection in Data Streams, Proc. 9th ACM SIGKDD International Conference on Knowkedge Discovery and Data Mining, pp , ACM (2003). [13] Shasha, D. and Zhu, Y.: High Performance Discovery in Time Series: Techniques and Case Studies, pp , New York University (2004). [14] Zhang, X. and Shasha, D.: Better Burst Detection, Proc. 22nd International Conference on Data Engineering, pp , IEEE (2006). [15] Vol.9, No.2, pp.1 6, (2010). [16] Vol.5, No.3, pp.86 96, (2012). [17] Shannon, C.: A Mathematical Theory of Communication, The Bell System Technical Journal, Vol.27, c 2013 Information Processing Society of Japan 83

16 Vol.6 No (Sep. 2013) pp , Bell Laboratories (1984). [18] Kullback, S. and Leibler, A.: On Information and Sufficiency, Annals of Mathematical Statistics, Vol.22, No.1, pp.79 86, Institute of Mathematical Statistics (1951). [19] Vol.2003, No.112, pp.1 7, (2003). [20] Kudo, K., Yamamoto, K. and Matsumoto, Y.: Applying Conditional Random Fields to Japanese Morphological Analysis, Proc Conference on Empirical Methods in Natural Language Processing, pp , ACL (2004) TIS UBC CAD/CG GIS/GPS Web CAD CAD ISO/TC184/SC4 CAD/ ISO CAD CAD Web Web Web c 2013 Information Processing Society of Japan 84