IPSJ-JNL

Size: px

Start display at page:

Download "IPSJ-JNL5212048"

はなありはら
7 years ago
Views:

1 Vol. 52 No (Dec. 2011) environment. Experimental results show a significant improvement in detection accuracy by the proposed method. Furthermore, subjective evaluations suggest that hot spots associated with these acoustic events are mostly useful, attracting the viewer s interest. Finally, we design a new interface podspotter, which provides efficient access to speech content based on these results = BIC GMM BIC Detecting Acoustic Events and Hot Spots Based on Audience s Reaction in Conversational Speech Content Tatsuya Kawahara, 1 Kouhei Sumi, 1 Jun Ogata 2 and Masataka Goto 2 We present a novel scheme for indexing hot spots in conversational speech content, such as podcasts, based on the reaction of the audience. Specifically, we focus on laughters and non-lexical reactive tokens, which are presumably related with funny spots and interesting spots, respectively. A robust detection method of these acoustic events is realized by combining BIC-based segmentation and GMM-based classification, with additional verifiers for reactive tokens. We also propose a novel method for automatically estimating and switching a penalty weight for the BIC-based segmentation according to the background acoustic 1. Web 4) = Web 1 School of Informatics, Kyoto University 2 National Institute of Advanced Industrial Science and Technology (AIST) 3363 c 2011 Information Processing Society of Japan

Finally, we design a new interface podspotter, which provides efficient access to speech content based on these results.

2 Fig. 1 Acoustic events and hot spots Web PodCastle 1 Google Audio Indexing PodCastle 15) Google Audio Indexing 1 1) 4),26) tf-idf 8) 2.2 Wrede 20) 2 Hot Spot Kennedy 10) Gatica-Perez 5) 2.3

Google Audio Indexing 1 http://podcastle.

3 GMM Gaussian Mixture Model HMM Hidden Markov Model SVM Support Vector Machines 2 Bayesian Information Criterion BIC 6),24) BIC 14),17) BIC BIC GMM BIC BIC GMM HMM 13),18) 12) SVM 11) Ward 19) yes 7) 3.1 BIC BIC BIC BIC 16) M 1,M 2,,M m {D 1,D 2,,D N } M i BIC BIC(M i)=logp (D 1,D 2,,D N M i) 1 λ di log N (1) 2 d i M i P M i BIC BIC 2),3) N 1 M 0 = N(μ 0, Σ 0) BIC BIC(M 0) j 1 <j<n 2 M 12 = {M 1,M 2} = {N(μ 1, Σ 1), N(μ 2, Σ 2)} BIC BIC(M 12) X = {x 1,,x N } M 0 : X = {x 1,,x N } N(μ 0, Σ 0) M 12 : {x 1,,x j} N(μ 1, Σ 1); {x j+1,,x N } N(μ 2, Σ 2) BIC(M 0)

4 3366 BIC(M 0)= d 2 N log 2π N 2 log Σ0 N λ ( d d(d +1) ) log N (2) d BIC(M 12) 2 BIC(M 12) = d 2 N log 2π j N j log Σ1 log Σ N λ (d + 1 ) 2 d(d +1) log N (3) ΔBIC(j) ΔBIC(j) =BIC(M 12) BIC(M 0) = 1 (N log Σ0 j log Σ1 (N j)log Σ2 ) 2 1 ( 2 λ d + 1 ) 2 d(d +1) log N (4) λ j =argmaxδbic(j) > 0 (5) j j λ 3) GMM GMM λ GMM GMM 1 λ BIC λ M GMM G m 2 + G m1 G m2 ΔBIC (4) 0 ΔBIC = 1 2 ((ng m1 + n Gm2 )log Σ Gm n Gm1 log Σ Gm1 n Gm2 log Σ Gm2 ) 1 ( 2 λm d + 1 ) 2 d(d +1) log(n Gm1 + n Gm2 ) 0 (6) m =1,,M Σ Gm Σ Gm1 Σ Gm2 n Gm1 n Gm2 EM G m1 G m2 EM m =1,,M (6) ΔBIC 0 λ m λ λ = 1 M M λ m (7) m= ) 25)

5 3367 (1) BIC t GMM θ (2) 23) (3) 4. 1 Table 1 Training data set for acoustic events. JNAS RWC RWC-MDB JNAS RWC-MDB JNAS IMADE 9) Web IMADE 4.1 = BIC GMM λ spe λ mix λ mus BIC 8 GMM 8 GMM 1 GMM MFCC ΔMFCC Δ 26 2 Fig. 2 Flow of acoustic event detection. 16 khz 25 ms 10 ms GMM BIC (1) W min (2) ΔBIC > 0

2 3 GMM λ spe λ mix λ mus BIC 8 GMM 8 GMM 1 GMM 256 3.1.2 2 12 MFCC ΔMFCC Δ 26 2 Fig.

6 3368 (3) (4) (2) (3) W min = GMM t =1.8 θ = GMM program-open 19 program-closed 23 λ spe λ mix λ mus λ spe λ mus λ mix 4.1 λ λ = R P F F F Table 2 Fig Frame-wise classification accuracy of 8-class acoustic events. 2 program-open Detection performance of laughters and reactive tokens (program-open). F λ = λ = λ = F = (1 + α2 )RP R + α 2 P F λ = λ = λ = α α = program-openprogram-closed λ 10% 2 λ =2.0 (8)

2 program-open Detection performance of laughters and reactive tokens (program-open). F 0.650 0.713 0.687 λ =1.0 0.913 0.264 0.305 λ =1.5 0.742 0.422 0.459 λ =2.0 0.600 0.

7 ) 46 1 BIC N max D max N max 20 D max Table 3 Questionnaire for hot spot evaluation. Q1 / Q2 / // / Q3 // / Q1 / Q2 // / / Q3 // / GUI 3 Q1 Q2 Q3 Q Q1 4 Q1 81.4% 89.4% 90% N max D max

8 Table 4 Detection performance of hot spots. Q1 / 81.4% 345/ % 338/ % 143/ % 133/147 / 4 Q2 Fig. 4 Result of Q2 for funny spots. Q2 4 5 Q3 6 7 Q2 5Q1 9 Q3 7 Q2 4 7 Q Q2 Fig. 5 Result of Q2 for interesting spots. 6 Q3 Fig. 6 Result of Q3 for funny spots. 6. Podspotter 8 Podspotter Q3 Fig. 7 Result of Q3 for interesting spots. Podspotter

Q2 4 5 Q3 6 7 Q2 5Q1 9 Q3 7 Q2 4 7 Q3 6 8 5 Q2 Fig. 5 Result of Q2 for interesting spots.

9 Podspotter C IT JST CREST 8 Podspotter Fig. 8 Outlook of Podspotter. Podspotter Adobe Flex ActionScript Flash Web OS 7. BIC GMM GMM BIC 1) Alberti, C., Bacchiani, M., Bezman, A., Chelba, C., Drofa, A., Liao, H., Moreno, P., Power, T., Sahuguet, A., Shugrina, M. and Siohan, O.: An Audio Indexing System for Election Video Material, Proc. IEEE-ICASSP, pp (2009). 2) Cettolo, M., Vescovi, M. and Rizzi, R.: Evaluation of BIC-based Algorithms for Audio Segmentation, Computer Speech and Language, Vol.19, No.2, pp (2005). 3) Chen, S. and Gopalakrishnan, P.: Speaker, Environment and Channel Change Detection and Clustering via the Bayesian Information Criterion, DARPA Broadcast News Workshop, pp (1998). 4) Furui, S. and Kawahara, T.: Transcription and Distillation of Spontaneous Speech, Springer Handbook on Speech Processing and Speech Communication, Benesty, J., Sondhi, M.M. and Huang, Y. (Eds.), pp , Springer (online) (2008), available from 5) Gatica-Perez, D., McCowan, I., Zhang, D. and Bengio, S.: Detecting Group Interest-Level in Meetings, Proc. IEEE-ICASSP, Vol.1, pp (2005). 6) Gauvain, J., Lamel, L. and Adda, G.: The LIMSI Broadcast News Transcription System, Speech Communication, Vol.37, No.1-2, pp (2002).

, Vescovi, M. and Rizzi, R.: Evaluation of BIC-based Algorithms for Audio Segmentation, Computer Speech and Language, Vol.19, No.2, pp.147 170 (2005). 3) Chen, S. and Gopalakrishnan, P.

10 3372 7) Gravano, A., Benus, S., Hirschberg, J., Mitchell, S. and Vovsha, I.: Classification of Discourse Functions of Affirmative Words in Spoken Dialogue, Proc. INTER- SPEECH, pp (2007). 8) Kawahara, T., Hasegawa, M., Shitaoka, K., Kitade, T. and Nanjo, H.: Automatic Indexing of Lecture Presentations using Unsupervised Learning of Presumed Discourse Markers, IEEE Trans. Speech & Audio Process., Vol.12, No.4, pp (2004). 9) Kawahara, T., Setoguchi, H., Takanashi, K., Ishizuka, K. and Araki, S.: Multi- Modal Recording, Analysis and Indexing of Poster Sessions, Proc. INTERSPEECH, pp (2008). 10) Kennedy, L. and Ellis, D.: Pitch-based Emphasis Detection for Characterization of Meeting Recordings, Proc. IEEE Workshop Automatic Speech Recognition and Understanding (ASRU03 ), pp (2003). 11) Kennedy, L. and Ellis, D.: Laughter Detection in Meetings, NIST Meeting Recognition Workshop (2004). 12) Knox, M.T. and Mirghafori, N.: Automatic Laughter Detection Using Neural Networks, Proc. INTERSPEECH, pp (2007). 13) Laskowski, K.: Contrasting Emotion-bearing Laughter Types in Multiparticipant Vocal Activity Detection for Meetings, Proc. IEEE-ICASSP, pp (2009). 14) Nishida, M. and Kawahara, T.: Speaker Model Selection based on the Bayesian Information Criterion applied to Unsupervised Speaker Indexing, IEEE Trans. Speech & Audio Process., Vol.13, No.4, pp (2005). 15) Ogata, J., Goto, M. and Eto, K.: Automatic Transcription for a Web 2.0 Service to Search Podcasts, Proc. INTERSPEECH, pp (2007). 16) Schwarz, G.: Estimating the Dimension of a Model, The Annals of Statistics, Vol.6, No.2, pp (1978). 17) Tranter, S. and Reynolds, D.: An Overview of Automatic Speaker Diarisation Systems, IEEE Trans. Audio, Speech, & Language Processing, Vol.14, pp (2006). 18) Truong, K.P. and Leeuwen, D.: Automatic Detection of Laughter, Proc. INTER- SPEECH, pp (2005). 19) Ward, N.: Pragmatic Functions of Prosodic Features in Non-Lexical Utterances, Speech Prosody, pp (2004). 20) Wrede, B. and Shriberg, E.: Spotting Hot Spots in Meetings: Human Judgments and Prosodic Cues, Proc. EUROSPEECH, pp (2003). 21) 15 pp (2009). 22) Vol.48, No.12, pp (2007). 23) Vol.83-D-II, No.11, pp (2000). 24) BIC SLP-82-6 (2010). 25) SLUD-A (2009). 26) Vol.91-D, No.2, pp (2008). ( ) ( ) ATR IEEESPSSpeechTCIEEE ASRU 2007 General Chair IEEE

Speech & Audio Process., Vol.12, No.4, pp.409 419 (2004). 9) Kawahara, T., Setoguchi, H., Takanashi, K., Ishizuka, K. and Araki, S.

11 WISS WISS IPA IT 25

ホットスポット 1 音リアクションイベント BIC GMM 2 3 BIC GMM HMM 10) SVM 11) 12) 13) Bayesian Information Criterion BIC 14) BIC M = M 1, M 2,,

ホットスポット 1 音リアクションイベント BIC GMM 2 3 BIC GMM HMM 10) SVM 11) 12) 13) Bayesian Information Criterion BIC 14) BIC M = M 1, M 2,, 1 1 2 2 BIC GMM Acoustic Event Detection for Finding Hot Spots in Podcasts Kouhei Sumi, 1 Tatsuya Kawahara, 1 Jun Ogata 2 and Masataka Goto 2 This paper presents a method to detect acoustic events that