18301.dvi

Size: px

Start display at page:

Download "18301.dvi"

えりかよしなが
5 years ago
Views:

1 Cognitive Studies, 18(3), (Sep. 2011) 2 1 1, 2 3 The ability to recognize emotional states of others is a fundamental social skill. In this study, we investigated the extent to which complex emotions can be inferred from facial or vocal cues in speech. Several sentences were prepared that intended to appreciate, blame, apologize, or congratulate others. Japanese university students uttered these sentences with congruent or incongruent emotional states, and they were recorded with a video camera. The speakers friends and strangers were shown these videos in a single modality (face or voice only) and they were asked to rate the perceived emotional states of the speakers. The results showed that the raters discriminated congruent message conditions from incongruent message conditions, and that this discrimination largely depended on voice cues, rather than face cues. The results also showed that the effects of familiarity of target person modulated the way of inferring emotional states. These results suggest that we could detect subtle emotional nuances of others in spoken interaction, and that we use facial and vocal information in some different ways. Keywords: facial expression, vocal expression, complex emotional states, communication (Ekman, 1972) (Larsen, McGraw, & Cacioppo, 2001; Larsen, McGraw, Mellers, & Cacioppo, 2004; Miyamoto, Uchida, & Ellsworth, 2010) (DePaulo & Kashy, 1998; Gross, Richards, & John, 2006) Recognizing Complex Emotional Messages by Facial and Vocal Expressions: An Investigation Using Expressions in Dialogue Situations, by Mitsue Nomura, Masato Nunoi, and Sakiko Yoshikawa (1: Kyoto University Graduate School of Education, 2: Japan Society for the Promotion of Science, 3: Kyoto University Kokoro Research Center). (de Gelder & Vroomen 2000; Hietanen, Leppanen, Illi, & Surakka, 2004; Vroomen, Driver, & de Gelder, 2001) de Gelder & Vroomen (2000)

2 442 Cognitive Studies Sep (Bugental, 1974; Mehrabian, 1970) (Krumhuber & Manstead, 2009) (Ishii, Reyes, & Kitayama, 2003) (Ekman & Friesen, 1969; Ekman, 1992) (DePaulo, 1992) (DePaulo, Lindsay, Malone, Muhlenbruck, Charlton, & Cooper, 2003) (Zuckerman, DePaulo, & Rosenthal, 1981) (Chovil, 1991; Fridlund, 1991) 2 (Motley & Camden, 1988; Zuckerman, Hall, DeFrank, & Rosenthal, 1976) (Ishii, Reyes, & Kitayama, 2003; Tanaka, Koizumi, Imai, Hiramatsu, Hiramoto, & de Gelder, 2010a) (Ekman & Friesen, 1974; Brandt, Miller, & Hocking, 1981) Ellis, Jones, & Mosdell (1997)

3 Vol. 18 No (Gobbini & Haxby, 2007; Palermo & Rhodes, 2007) (1) (2) (3) (4) DV )

4 444 Cognitive Studies Sep (1) (2) a. b. (1) c. (2) (3) (4) a. b. (1) c. (2)

5 Vol. 18 No SD SD a. (F (1, 9) = 25.47, p <.001) (F (1, 9) = 9.86, p<.05) ( F (1, 9) = 6.14, 37.43; p<.05, p<.001) b. (F (1, 9) = 15.27, p<.01) c. (F (1, 9) = 11.83, p<.01) (F (1, 9) = 7.32, p<.05) (F (1, 9) = 28.58, p<.001) (ps >.10) a. (F (1, 9) = 19.04, p<.01) (p >.10) b. (F (1, 9) = 50.77, p<.001)

6 446 Cognitive Studies Sep ( SD) (1) a (1.80) 2.30 (1.27) 4.50 (1.02) 1.90 (1.14) 3.64 (0.75) 2.92 (1.03) 3.78 (0.62) 2.12 (0.58) b (1.95) 3.50 (1.50) 2.00 (0.77) 3.90 (1.64) 2.60 (0.96) 3.32 (0.52) 2.10 (0.91) 3.99 (0.60) c (1.36) 2.80 (1.89) 1.70 (0.90) 3.80 (1.47) 2.39 (0.93) 2.89 (0.64) 2.23 (0.60) 3.70 (0.73) (2) a (1.22) 2.70 (1.49) 3.70 (0.78) 2.40 (1.43) 3.58 (0.69) 2.59 (0.81) 3.74 (0.44) 2.62 (0.47) b (1.04) 3.60 (1.43) 1.60 (1.20) 4.00 (1.10) 2.59 (0.83) 3.18 (0.62) 1.78 (0.61) 3.36 (0.74) c (1.97) 3.80 (1.60) 2.10 (1.37) 4.00 (1.34) 2.29 (0.92) 3.27 (0.91) 1.81 (0.55) 3.47 (0.79) (3) a (1.14) 2.50 (1.80) 3.60 (0.92) 2.20 (0.98) 3.68 (0.67) 2.14 (1.01) 3.59 (0.49) 2.88 (0.78) b (0.98) 4.20 (1.72) 2.20 (1.66) 3.80 (1.33) 2.41 (0.53) 3.64 (1.06) 2.12 (0.75) 3.18 (0.80) c (1.49) 3.50 (1.96) 2.40 (1.43) 3.90 (1.30) 2.72 (0.70) 3.40 (0.83) 2.04 (0.87) 3.11 (0.76) (4) a (1.51) 2.50 (1.80) 3.30 (1.62) 2.90 (1.22) 3.41 (0.88) 2.78 (0.67) 3.57 (0.59) 3.20 (0.57) b (1.72) 4.30 (1.73) 2.30 (1.73) 3.50 (0.67) 2.81 (0.84) 3.80 (0.64) 2.61 (0.90) 3.32 (0.65) c (1.42) 3.50 (1.02) 2.70 (1.42) 3.70 (0.64) 2.60 (0.82) 3.30 (0.39) 2.62 (0.62) 3.30 (0.41) (F (1, 9) = 9.88, p<.05) (F (1, 9) = 28.58, p<.001) (F (1, 9) = 18.36, p<.01) c. (F (1, 9) = 11.82, p<.01) (F (1, 9) = 6.37, p<.05) (F (1, 9) = 36.86, p<.001) (F (1, 9) = 3.94, p<.10) (ps >.10) a. (F (1, 9) = 15.89, p<.01) (F (1, 9) = 3.76, p<.10) b. (F (1, 9) = 21.52, p<.01) c. (F (1, 9) = 13.38, p<.01)

7 Vol. 18 No (F (1, 9) = 13.09, p<.01) (F (1, 9) = 6.14, p<.05) (p >.10) a. (p >.10) b. (F (1, 9) = 24.11, p<.001) c. (F (1, 9) = 16.19, p<.01) (p >.10) 3 (Motley & Camden,1988; Zuckerman, Hall, DeFrank, & Rosenthal, 1976) (Ishii, Reyes, & Kitayama, 2003; Tanaka, Koizumi, Imai, Hiramatsu, Hiramoto, & de Gelder, 2010a) (Collingnon, Girard, Gosselin, Roy, Saint- Amour, Lassonde, & Lepore, 2008; Tanaka, Koizumi, Imai, Hiramatsu, Hiramoto, & de Gelder, 2010b) 1 Collingnon et al. (2008) Tanaka et al. (2010b) Collingnon et al. (2008) Tanaka et al. (2010b)

8 448 Cognitive Studies Sep (Frith, 2007) Nomura & Yoshikawa (2011)

9 Vol. 18 No (Russell, Bachorowski, & Fernandez Dols, 2003) (Campanella & Belin, 2007) (Zaki, Hennigan, Weber, & Ochsner, 2010; Zhu, Zhang, Wu, Luo, & Luo, 2010) Brandt, D. R., Miller, G. R., & Hocking, J. E. (1980). The truth deception attribution: Effects of familiarity on the ability of observers to detect deception. Human Communication Research, 6, Bugental, D. E. (1974). Interpretations of naturally occurring discrepancies between words and intonation: Modes of inconsistency resolution. Journal of Personality and Social Psychology, 30, Campanella, S., & Belin, P. (2007). Integrating face and voice in person perception. Trends in Cognitive Sciences, 11, Chovil, N. (1991). Social determinants of facial displays. Journal of Nonverbal Behavior, 15, Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., & Lepore, F. (2008). Audio-visual integration of emotion expression. Brain Research, 1242, de Gelder, B., & Vroomen, J. (2000). The perception of emotions by ear and by eye. Cognition and Emotion, 14, DePaulo, B. M. (1992). Nonverbal behavior and self-presentation. Psychological Bulletin, 111, DePaulo, B. M., & Kashy, D. A. (1998). Everyday lies in close and casual relationships. Journal of Personality and Social Psychology, 74, DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). Cues to deception. Psychological Bulletin, 129, Ekman, P. (1972). Universals and cultural differences in facial expressions of emotion. In J. Cole (Ed.), Nebraska Symposium on Motivation, 1971, (Vol. 19, pp ). Ekman, P. (1992). Telling lies. NewYork: Norton. Ekman, P., & Friesen, W. V. (1969). Nonverbal leakage and clues to deception. Psychiatry, 32, Ellis, H. D. Jones, D. M., & Mosdell, N. (1997). Intra- and inter-modal repetition priming of familiar faces and voices. British Journal of Psychology, 88, Fridlund, A. J. (1991). Sociality of social smiling: Potentiation by an implicit audience. Journal of Personality and Social Psychology. 60, Frith, C. D. (2007). The social brain? Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences, 362, Gobbini, M. I., & Haxby, J. V. (2007). Neural

10 450 Cognitive Studies Sep systems for recognition of familiar faces. Neuropsychologia, 45, Gross, J. J., Richards, J. M., & John, O. P. (2006). Emotion regulation in everyday life. In D.K. Snyder., J.A. Simpson, & J.N. Hughes (Eds.), Emotion regulation in families: Pathways to dysfunction and health. Washington DC: American Psychological Association. (pp ). Hietanen, J. K., Leppanen, J. M., Illi, M., & Surakka, V. (2004). Evidence for the integration of audiovisual emotional information at the perceptual level of processing. European Journal of Cognitive Psychology, 16, Ishii, K., Reyes, J. A., & Kitayama S. (2003). Spontaneous attention to word content versus emotional tone: Differences among three cultures. Psychological Science, 14, Krumhuber, E., & Manstead, A. S. R. (2009). Are you joking? The moderating role of smiles in the perception of verbal statements. Cognition and Emotion, 23, Larsen, J. T., McGraw, A. P., & Cacioppo, J. T. (2001). Can people feel happy and sad at the same time? Journal of Personality and Social Psychology, 81, Larsen, J. T., McGraw, A. P., Mellers, B. A., & Cacioppo, J. T. (2004). The agony of victory and thrill of defeat: Mixed emotional reactions to disappointing wins and relieving losses. Psychological Science, 15, Mehrabian, A. (1970). When are feelings communicated inconsistently? Journal of Experimental Research in Personality, 4, Miyamoto, Y., Uchida, Y., & Ellsworth, P. C. (2010). Culture and mixed emotions: Cooccurrence of positive and negative emotions in Japan and the U.S. Emotion, 10, Motley, M. T., & Camden, C. T. (1988). Facial Expression of Emotion: A comparison of posed expressions versus spontaneous expressions in an interpersonal communication setting. Western Journal of Speech Communication, 52, Nomura, M., & Yoshikawa, S. (2011). Gaze and facial expressions when talking about emotional episodes. Psychologia, 54, Palermo, R., & Rhodes, G. (2007). Are you always on my mind? A review of how face perception and attention interact. Neuropsychologia, 45, Russel, J. A., Bachorowski, J. A., & Fernandez- Dols, J. M. (2003). Facial and vocal expressions of emotion. Annual Review of Psychology, 54, Tanaka, A., Koizumi, A., Imai, H., Hiramatsu, S., Hiramoto, E., & de Gelder, B. (2010a). I feel your voice: Cultural differences in the multisensory perception of emotion. Psychological Science, 21, Tanaka, A., Koizumi, A., Imai, H., Hiramatsu, S., Hiramoto, E., & de Gelder, B. (2010b). Cross-Cultural Differences in the Multisensory Perception of Emotion. Proceedings of the International Conference on Auditory- Visual Speech Processing 2010, Vroomen, J., Driver, J., & de Gelder, B. (2001). Is cross-modal integration of emotional expressions independent of attentional resources? Cognitive, Affective and Behavioral Neuroscience, 1, Zaki, J., Hennigan, K., Weber, J., & Ochsner, K. (2010). Social cognitive conflict resolution: The role of domain general and domain specific neural systems. Journal of Neuroscience, 30, Zhu, X. R., Zhang, H. J., Wu, T. T., Luo, W. B., & Luo, Y. J. (2010). Emotional conflict occurs at an early stage: Evidence from the emotional face-word Stroop task. Neuroscience Letters, 478, 1 4. Zuckerman, M., DePaulo, B. M., & Rosenthal, R. (1981). Verbal and nonverbal communication of deception. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 14, pp. 2 60). San Diego, CA: Academic Press. Zuckerman, M., Hall, J. A., DeFrank, R. S., & Rosenthal, R. (1976). Encoding and decoding of spontaneous and posed facial expressions. Journal of Personality and Social Psychology, 34, (Received 1 March 2011) (Accepted 16 June 2011)

11 Vol. 18 No a. ( ) 7 1 b. ( ) 7 1 c. ( ) 7 1

12 452 Cognitive Studies Sep ERATO DC2 ()

18301.dvi

18301.dvi Cognitive Studies, 18(3), 416-427. (Sep. 2011) Information from face and voice plays an important role in social communication. As shown in the study of speech perception, facial and vocal signals are