163 Original Paper Music Retrieval Based on the Relation between Color Association and Lyrics Tetsuaki Nakamura Akira Utsumi Maki Sakamoto Graduate School of Informatics and Engineering, The University of Electro-Communications nakamura@utm.se.uec.ac.jp utsumi@inf.uec.ac.jp, http://www.utm.se.uec.ac.jp/ utsumi/ sakamoto@hc.uec.ac.jp, http://www.sakamoto-lab.hc.uec.ac.jp/ keywords: music, color, lyric, recommendation, retreival Summary Various methods for music retrieval have been proposed. Recently, many researchers are tackling developing methods based on the relationship between music and feelings. In our previous psychological study, we found that there was a significant correlation between colors evoked from songs and colors evoked only from lyrics, and showed that the music retrieval system using lyrics could be developed. In this paper, we focus on the relationship among music, lyrics and colors, and propose a music retrieval method using colors as queries and analyzing lyrics. This method estimates colors evoked from songs by analyzing lyrics of the songs. On the first step of our method, words associated with colors are extracted from lyrics. We assumed two types of methods to extract words associated with colors. In the one of two methods, the words are extracted based on the result of a psychological experiment. In the other method, in addition to the words extracted based on the result of the psychological experiment, the words from corpora for the Latent Semantic Analysis are extracted. On the second step, colors evoked from the extracted words are compounded, and the compounded colors are regarded as those evoked from the song. On the last step, colors as queries are compared with colors estimated from lyrics, and the list of songs is presented based on similarities. We evaluated the two methods described above and found that the method based on the psychological experiment and corpora performed better than the method only based on the psychological experiment. As a result, we showed that the method using colors as queries and analyzing lyrics is effective for music retrieval. 1. 1 [ 06, 07, 06] [ 06] SD [ 01] [ 06, 01, 03, 03, 05] [ 03] [ 94]
164 27 3 D 2012 [ 01] [Bernard 86] [ 03] [Block 83] [ 07] 2 1 2 1 2 [ 02, 05] [ 03] [ 07] [ 11] [ 11] 2. 2 1 1 35 1 v c i (i =1,...,35) p i (1) v =(p 1,...,p 35 ) (1) q m v(m) q v(m) s(q,v(m)) (2) 1 1 Microsoft Word2003 40 5
165 s(q,v(m)) = 1 q v(m) q v(m) v(m) 2 2 [ 11] (3) m v(m) w v(m)= i A(m) f(w i)i(w i )v(w i ) (3) A(m) A(m) m f(w i ) m w i A(m) I(w i ) w i v(w i ) w i w i w i 2 3 (3) 1 80 A 60 47 13 20.78 (2) 80 30 2 40 40 1 30 1 1 / 2 3 4 2 2 3 1 a r(a) (4) m r(a)= i M(a) r(m i,a) (4) M(a) r(m i,a)= x(m i,a) x(m i ) M(a) a x(m i ) m i M(a) x(m i,a) m i m i a 2 3 1 m i 2 [ 11] 1 2 3 CD 4
166 27 3 D 2012 x(m i )=30 80 3 3 27 25 29 r( ) (27/30 + 25/30 + 29/30)/3= 0.9 (4) r(a) r(a) 5/30 369 283 76.69% 86 23.31% r(a) 5/30 369 r(a) 15/30 70 66 94.29% 4 5.71% i.e., (4) r(a) 30 B r(a) 5/30 (3) 283 3 (4) 2 3 2 5/30 5/30 4 2 3 1 2 3 1 21 17 4 23.14 2 1 PC 3 1 2 2 2 3 2 1 w v(w) (5) v(w)=( t(w,c 1) 3x(w),..., t(w,c 35) 3x(w) ) (5) x(w) w t(w,c i ) w c i (i = 1,...,35) x(w)=21 2 4 2 3 2 3 1 2 3 1 (3) LSA [Landauer 07] LSA 2005
167 1 2 3 80 LSA LSA LSA 1 1 1 MeCab 5 LSA LSA 643,807 129,462 300 [Landauer 97] LSA 1 u θ(> 0) u u u I(u) v(u) (6) (8) I(u)= w i P (u,θ) s(u,w i)i(w i ) P (u,θ) (6) v(u)= w i P (u,θ) s(u,w i)v(w i ) P (u,θ) (7) P (u,θ)={w i w i P s(u,w i ) θ} (8) P I(w i ) w i P v(w i ) w i s(u,w i ) LSA u w i P (u,θ) u s(u,w i ) θ w i P (u,θ) u 0 2 5 (3) 2 1 (3) 5 http://mecab.sourceforge.net/ 3 A(m) 1 1 (3) A(m) 2 4 2 (3) 2 3 2 4 2 6 3 3 35 1 1 35 3 3 4 3. 3 1 i.e., [ 11]
168 27 3 D 2012 1 θ correct 0.6 0.7 0.8 0.9 1.0 16.48 10.12 5.44 2.08 1.00 4 F [ 02] 1 2 3 50 2 3 C 2 3 1 2 3 4 57 27 30 20.10 50 1 2 3 2 3 4 m i (i =1,...,50) q(m i ) (9) q(m i )=( t(m i,c 1 ) 3x(m i ),..., t(m i,c 35 ) 3x(m i ) ) (9) x(m i ) m i t(m i,c j ) m i c j (j =1,...,35) x(m i )=57 2 (10) (12) R P F R P R = (10) P = (11) F = 2 1/R +1/P (12) θ correct θ correct =1.0 θ correct < 1.0 (3) N θ correct 0.6 0.7 0.8 0.9 1.0 N 1 10 (6) (8) θ 0.3 0.4 0.5 0.6 0.7 θ correct 1 3 (3) A(m) 1 2 2 (12) F (6) (8) θ 3 1 2 5 0.3 0.4 0.5 0.6 0.7 θ correct F θ =0.6 2 5 14 N 50 i.e., [ 99] 5 θ correct =1.0 5 N
169 5 θ correct =1.0 7 θ correct =0.8 6 θ correct =0.9 8 θ correct =0.7 2 1 6 9 θ correct = 0.9 0.8 0.7 0.6 6 9 N 2 1 (10) 5 9 1 2 10 θ correct =1.0 10 N N 5 2 1 N N >5 11 14 θ correct = 0.9 0.8 0.7 0.6 11 14 N N 5 2 1 N N >5 (11)
170 27 3 D 2012 9 θ correct =0.6 11 θ correct =0.9 10 θ correct =1.0 12 θ correct =0.8 10 14 N 2 1 1 2 3 2 2 (6) (8) θ 0.6 Pearson m i (i =1,...,50) m i c j (j =1,...,35) x ij m i i.e., c j y ij m i x y Pearson 35 0.3338 50 0.3338 i.e., 50 37 74% 2
171 3 1 3 1 2 2 3 1 4 2 13 θ correct =0.7 1 14 θ correct =0.6 4. 4 1 2 3 1 80 2 3 1 2 3 4 3 1 1 20 17 3 23.45 10 2 40 40 1 10 2 3 1
172 27 3 D 2012 2 m i (i =1,...,80) r(m i ) (13) r(m i )= P (m i) A(m i ) P (m i ) (13) P (m i ) m i A(m i ) m i 2 m i r(m i ) 0.7511 0.1912 4 3 130 80 50 1970 1980 1990 2000 J-POP 3 4 2 5. i.e., [Bernard 86] Bernard, J. W.: Messiaen s Synaesthesia: The Correspondence between Color and Sound Structure in His Music, Music Perception, Vol. 4, No. 1, pp. 41 68 (1986) [Block 83] Block, L.: Comparative Tone-Colour Responses of College Music Majors with Absolute Pitch and Good Relative Pitch, Psychology of Music, Vol. 11, No. 2, pp. 59 66 (1983) [ 01],, (2001) [ 07], VocalFinder,, Vol. 2007, No. 81, pp. 27 32 (2007) [ 06],,,,, Vol. 2006, No. 133, pp. 19 24 (2006) [ 01],,,, Vol. 42, No. 12, pp. 3201 3212 (2001) [ 03],,, Vol. 27, No. 22, pp. 5 8 (2003) [ 02],,,, (2002) [ 03],,, Vol. 103, No. 407, pp. 1 6 (2003) [ 06],,, Vol. 21, No. 3, pp. 310 318 (2006) [ 03],,,
173, Vol. 103, No. 521, pp. 41 44 (2003) [ 94], Colour Coordinator,, Vol. 1994, No. 30, pp. 89 96 (1994) [Landauer 97] Landauer, T. K. and Dumais, S. T.: A Solution to Plato s Problem: The Latent Semantic Analysis Theory of the Acquisition, Induction, and Representation of Knowledge, Psychological Review, Vol. 104, No. 2, pp. 211 240 (1997) [Landauer 07] Landauer, T. K., McNamara, D. S., Dennis, S., and Kintsch, W. eds.: Handbook of Latent Semantic Analysis, Lawrence Erlbaum Associates, London (2007) [ 03],,,,, (A), Vol. J86 A, No. 11, pp. 1219 1230 (2003) [ 11],,, (A), Vol. J94-A, No. 2, pp. 85 94 (2011) [ 06],,,,, Vol. 2006, No. 113, pp. 3 8 (2006) [ 07],, 24, pp. 184 187 (2007) [ 05],,,, Vol. 46, No. 7, pp. 1560 1570 (2005) [ 99],, (1999) [ 02],,, Vol. 2002, No. 100, pp. 105 109 (2002) [ 05],,, Vol. 5, No. 3, pp. 31 37 (2005) 2011 6 15 A. 2 3 A.1 B. i.e., (4) 30 B.2 B.2 30 0.95 0.70 0.93 0.70 0.93 0.70 0.93 0.68 0.90 0.67 0.90 0.67 0.90 0.67 0.83 0.67 0.80 0.65 0.77 0.64 0.76 0.63 0.73 0.63 0.73 0.63 0.71 0.63 0.70 0.63 2011 Cognitive Science Society 1993 2010 Cognitive Science Society 1998 2010 Cognitive Science Society C. 3 C.3
174 27 3 D 2012 A.1 DREAMS COME TRUE DAYDREAM JUDY AND MARY C EST LA VIE CHEMISTRY No limit Every Little Thing PURE KinKi Kids BUMP OF CHICKEN GReeeeN DREAMS COME TRUE TUBE TULIP ( ) KOKIA pray ELISA Every Little Thing THE ALFEE TUBE ebullient future (Japanese) ELISA Boyfriend Crystal Kay SHA-LA-LA I WiSH DREAMS COME TRUE ONE SURVIVE 1985 Factory Street RETURN TO THE MOON CONSTRUCTION NINE TIP ON DUO TUBE JUDY AND MARY snow drop Happy Xmas with ( ZARD ) IWiSH K3 TUBE monochrome Hello! Orange Sunshine JUDY AND MARY ZONE Romeo Cosmic Picnic THE ALFEE TUBE Spring has come rino SUNNY DAY MISIA Answer December I WiSH Only You TUBE KinKi Kids FOREVER YOURS Every Little Thing Shapes Of Love Every Little Thing ZARD ZARD ZARD Every Little Thing Every Little Thing the brilliant green SUMMER DREAM TUBE Blame L Arc en Ciel IWiSH Mr.Children GReeeeN
175 C.3 KinKi Kids DREAMS COME TRUE mihimaru GT Sunny Day Sunday Whiteberry BUMP OF CHICKEN IWiSH DAYS FLOW HOME MADE Rain YUKI FANTASY Chara 12 JUDY AND MARY DEPARTURES globe appears HIGH PRESSURE T.M.Revolution Ho! & SMILY Let s go faraway Sowelu PUFFY BODY & SOUL SPEED HOTEL PACIFIC SMAP Winter,again GLAY Point of No Return CHEMISTRY Butterfly LIFE is another story THE BOOM PRIDE HIGH and MIGHTY COLOR Cocco UA