Vol. 47 No. 6 June 2006 non-modal non-modal Acoustic Representation of Prosodic and Voice Quality Features and their Relationship with Perception of P

Size: px

Start display at page:

Download "Vol. 47 No. 6 June 2006 non-modal non-modal Acoustic Representation of Prosodic and Voice Quality Features and their Relationship with Perception of P"

みいかいいはた
5 years ago
Views:

1 Vol. 47 No. 6 June 2006 non-modal non-modal Acoustic Representation of Prosodic and Voice Quality Features and their Relationship with Perception of Paralinguistic Information in Dialog Speech Carlos Toshinori Ishi, Hiroshi Ishiguro and Norihiro Hagita To date, most works dealing with paralinguistic information extraction have focused only on prosodic features like fundamental frequency (F0), power and duration. However, when analyzing natural conversational speech data, the presence of several voice qualities (caused by non-modal phonations) is often observed, mainly in expressive speech utterances. In the present work, the use of voice quality features in addition to classical prosodic features is proposed for automatic extraction of paralinguistic information (intentions, attitudes and emotional expressions) in dialog speech. Perceptual experiments and acoustic analyses are conducted for monosyllabic utterances spoken in several speaking styles (acted) in order to produce different paralinguistic information. Acoustic parameters related with prosodic and voice quality features potentially representing the variations in speaking styles are evaluated. Experimental results indicate that prosodic features are effective for identifying some groups of paralinguistic information carrying specific functions, while voice quality features are useful for identifying utterances with an emotional or attitudinal expressivity. Experiments are also conducted on natural conversational speech data with emphasis on utterances containing nonmodal voice qualities. Results of these natural speech data showed the same trends as those of the acted ones. Evaluation results on the proposed algorithms for automatic detection of each voice quality are also reported. 1. ATR ATR Intelligent Robotics and Communication Laboratories { } 1) 4) { 1782

2 Vol. 47 No } 5) { } 6) 7) / / / / / / 5) 6) F0 prosodic features 4) 6),8) 2),3),8) 9) 12) non-modal F0 13) voice quality 14) laryngeal voice quality 14) modal breathy whispery vocal fry creaky harsh ventricular 1) { } 15) / 16) 19) 1 1 Fig. 1 Framework for paralinguistic information extraction including the proposed speaking style representation using prosodic and voice quality features non-modal

3 1784 June Table 1 List of paralinguistic information for the utterance e. 0?!?! CREST/ESP 20) { } { } { } 1 2 A B 21)

4 Vol. 47 No Table 2 Matches, mismatches and ambiguities between intended and perceived paralinguistic items. 2 Fig. 2 Grouping of the paralinguistic information items according to the perceptual data results. 3 Table 3 Groups of perceived paralinguistic information used for acoustic analysis ) modal m whispery w aspirated a creaky c harsh h pressed p hw pc 3

5 1786 June Fig. 3 Distribution of the perceived voice qualities, for each perceived paralinguistic information group. 3 w a 3 3 non-modal h hw a w pc w 23) 3 m modal non-modal non-modal F0 LPC non-modal 4 Fig. 4 Distributions of the prosodic parameters for each perceived paralinguistic information group. F0 sub-harmonic 17) 16) F0move F0move semitone 2 F0 16) F0 F0avg2a F0tgt2b F0move=F0tgt2b F0avg2a F0move F0 F0 16) 20 db duration 4 F0move vs. duration

6 Vol. 47 No Fig. 6 Simplified block diagram of the parameters for aperiodicity/double-periodicity detection. 5 Vocal fry Fig. 5 Simplified block diagram of the vocal fry detection. F0 harsh creaky F0 F0move non-modal F Vocal fry creaky 17) vocal fry creaky vocal fry ms 5 10 ms 5ms 2.5 ms very short-term 5 vocal fry 3 PPw: Peak Power IFP: Intra-Frame Periodicity IPS: Inter-Pulse Similarity 17) PPw > 7dB IFP < 0.8 IPS > aperiodicity/ double-periodicity Vocal fry harsh 18) creaky vocal fry harsh vocal fry harsh NACR: Normalized Auto-Correlation Ratio 2 2 TLR: Time-Lag Ratio NACR > > TLR > ) aspiration noise breathy whispery turbulent noise breathy whispery 14)

7 1788 June Fig. 7 Simplified block diagram of the parameters for aspiration noise detection. 24) harsh harsh whispery voice 14) 19) F1 F3 F1F3syn: F1 and F3 band synchronization F1 F3 A1 A3 F1F3syn F1 F3 7 F1F3syn A1 A3 F1F3syn A1 A3 F3 F1 F3 F ,500 Hz F3 1,800 4,500 Hz 19) F1F3syn < 0.4 A1 A3 < 25 db Vocal Fry Rate V FR vocal fry creaky Aperiodicity Rate AP R vocal fry Aspiration Noise Rate ANR V F vocal fry AP 8 Fig. 8 Distribution of the detected voice qualities, for each perceived paralinguistic information group. ANM modal 4 V F AP AN M pc c h hw w a m 0.1 VFR > 0.1 VF APR > 0.1 AP ANR > 0.1 AN M 8 8 AP AN AP AN AN 3 AP h hw harsh V F 8 3 pc creaky c creaky 4. non-modal 2

8 Vol. 47 No Table 4 Confusion matrix between perceived and detected voice qualities. 9 nonmodal Fig. 9 Distribution of the perceived paralinguistic information groups for each perceived non-modal voice qualities (Natural speech data). non-modal 2 non-modal non-modal CREST/ESP Expressive Speech Database 20) 30 1 FAN 3 1 non-modal 2.3 modal non-modal {pc = VF (15) h,hw = AP (15) w = AN (30)} 22) w = AN h,hw = AP pc = VF 9 non-modal 4 w = AN vocal fry pc = VF9 harsh AP M VF harsh harsh 5. non-modal vocal fry harsh SVM

9 1790 June 2006 NTT ATR/HIS ATR/HIS IRC 1) Fernandez, R. and Picard, R.W.: Classical and Novel Discriminant Features for Affect Recognition from Speech, Proc. Interspeech 2005, pp (2005). 2) Schuller, B., Muller, R., Lang, M. and Rigoll, G.: Speaker Independent Emotion Recognition by Early Fusion of Acoustic and Linguistic Features within Ensembles, Proc.Interspeech 2005, pp (2005). 3) 2005 Vol.I, pp (2005). 4) KL 2005 Vol.I, pp (2005). 5) / Vol.J88- D-II, No.3, pp (2005). 6) 2004 Vol.I, pp (2005). 7) Vol.61, No.12, pp (2005). 8) 2005 Vol.I, pp (2005). 9) Erickson, D.: Expressive speech: production, perception and application to speech synthesis, Acoust. Sci. & Tech., Vol.26, No.4, pp (2005). 10) Maekawa, K.: Production and perception of Paralinguistic information, Proc. Speech Prosody 2004, pp (2004). 11) Klasmeyer, G. and Sendlmeier, W.F.: Voice and Emotional States, Voice Quality Measurement, Ch.15, pp , Singular Thomson Learning (2000). 12) Gobl, C. and Ní Chasaide, A.: The role of voice quality in communicating emotion, mood and attitude, Speech Communication, Vol.40, pp (2003). 13) Hess, W.: Pitch Determination of Speech Signals, Vol.3 of Springer Series of Information Sciences, Springer-Verlag, Berlin, Heidelberg, New York (1983). 14) Laver, J.: Phonatory settings, The phonetic description of voice quality, Ch.3, pp , Cambridge University Press (1980). 15) 2005 Vol.I, pp (2005). 16) Ishi, C.T.: Perceptually-related F0 parameters for Automatic Classification of Phrase Final Tones, IEICE Trans. Inf. & Syst., Vol.E88-D, No.3, pp (2005). 17) Ishi, C.T., Ishiguro, H. and Hagita, N.: Proposal of Acoustic Measures for Automatic Detection of Vocal Fry, Proc. Eurospeech 2005, pp (2005). 18) Ishi, C.T.: Analysis of Autocorrelation-based parameters for Creaky Voice Detection, Proc. Speech Prosody, pp (2004). 19) Ishi, C.T.: A New Acoustic Measure for Aspiration Noise Detection, Proc. ICSLP 2004, Vol.II, pp (2004). 20) 21) Sadanobu, T.: A Natural History of Japanese Pressed Voice, J. Phonetic Society of Japan, Vol.8, No.1, pp (2004). 22) carlos/voicequality/ 23) Ito, M.: Politeness and voice quality The alternative method to measure aspiration noise, Proc. Speech Prosody 2004, pp (2004). 24) Kreiman, J. and Gerratt, B.: Measuring Vocal Quality, Voice Quality Measurement, Ch.7, pp , Singular Thomson Learning (2000). A? B A? B A B A B A

Vol. 47 No. 6 1791 B A B A? B A? B A rainy B?? A7 B?? A B A B A B! A B! A B A B!? A B! A B? A 1!

B ( 17 10 17 ) ( 18 4 4 ) 1996 ITA Instituto Tecnológico de Aeronáutica 1998 1998

10 Vol. 47 No B A B A? B A? B A rainy B?? A7 B?? A B A B A B! A B! A B A B!? A B! A B? A 1! B A B A! B! A B! A! B! A B A B! A! B? A 3 B A B A ? B A ? B ( ) ( ) 1996 ITA Instituto Tecnológico de Aeronáutica JST/CREST ESP ATR 2005 ATR ATR ATR

11 1792 June NTT NTT ATR ATR IEEE

untitled

untitled JAIS 1 2 1 2 In this paper, we focus on the pauses that partly characterize the utterances of simultaneous interpreters, and attempt to analyze the results of experiments conducted using human subjects