IPSJ SIG Technical Report On a Bayesian Network-based Model for Referring Expressions Kotaro Funakoshi, 1 Mikio Nakano, 1 Takenobu Tokunaga 2

Size: px

Start display at page:

Download "IPSJ SIG Technical Report On a Bayesian Network-based Model for Referring Expressions Kotaro Funakoshi, 1 Mikio Nakano, 1 Takenobu Tokunaga 2"

よりとしままだ
4 years ago
Views:

1 On a Bayesian Network-based Model for Referring Expressions Kotaro Funakoshi, 1 Mikio Nakano, 1 Takenobu Tokunaga 2 and Ryu Iida 2 A Bayesian network-based model available both for resolution and generation of referring expressions in dialogue systems is presented. The model handles descriptions, anaphoras, and deixes in a unified way. This paper describes the overview and use of the model, and shows in what manner various expressions are handled as Bayesian networks. Finally integration of the model to spoken dialogues systems and related work are discussed Honda Research Institute Japan Co., Ltd. 2 Tokyo Institute of Technology WCXD 1 WCXD 2 3 one-anaphora interactive alignment 6)

2 W C X D 1 WCXD 6 2. REBN (Referring Expression Bayesian Network) V D(V ) ) Burger Connolly 2) Weissenbacher 19) it 9) 20) WCXD 1 REBN WCXD 1 WCXD WCXD W, C, X, D W C X D 5),15) D(X) D X X C W 4 W C X X D W C X 2 1 WCXD 1 1 REBN P (W, C, X, D) = P (W C, X)P (C X, D)P (X D)P (D) (1) REBN x x W P (W, C, X, D) P (X = x W ) REBN W 1 W 2 REBN P (W 1, W 2, C 1, C 2, X, D) 1 P (X W 1, W 2 ) REBN X W D C W 1 W 2 P (W 1, W 2, C 1, C 2, X 1, X 2, D 1, D 2 ) P (X 2 W 1, W 2) C 3 another 1 2

3 W1 W2 C1 C W1 C1 X1 D1 W2 C2 X2 D (W -X C-D ) P (W C, X), P (C X, D), P (X D), P (D) REBN 2.3 REBN x ( 1 ) REX-graph REX-graph ( 2 ) REX-graph X D REBN ( 3 ) REBN REBN P (X E) x E REBN W i E = {W 1,..., W n} x = argmax x D(X) P (x e) (2) e E REX-graph REX-graph REX-graph A:( ) A A,B A B A B B A 2 REBN REX-graph A:( ) B:( ) A B 3 REBN REX-graph A:( ) B:( ) C:( ) D:( ) A B B C C D 3

4 A:( ) B:( ) C:( ) D:( ) A C B C C D REX-graph REX-graph (i, x ) = argmax i I,x D(X) P i (x e) (3) 1 I P i (X E) i REBN 2.4 REBN generate-and-test 2 REBN 1 REBN (1) (2) (3) (A) (B) (C) 3 (A) E REBN P e (X E = e) x 1 i 2 REBN REBN X,D 3 incremental algorithm 4) e = argmax e E P e (X = x E = e) (4) P (Ê) e = argmax e E P e (X = x E = e)p (Ê = e) (5) REBN P (D) d d D(D) 1 P (D) d 0 d 1 2 d 2 D(D) D(D) Denis 5) P (D) P (X D) P (X = x D = d) d x D(X) D(D) D(X) = d x d P (x d) = 0 P (X D) 23) 4 D(Ê) = E D(E) E 5 P (X, Ê) 1 REBN P (Ê X) 4

5 X 2 D(D) P (C X, D) P (C = c X = x, D = d) x c c x relevancy c x D x d D(C) REBN REX-graph D(C) W C chair sofa table P (W C, X) P (W = w C = c, X = x) x c P (W C, X) P (W C) X P (W = C = name, X = human1) X P (W C, X) X X = human1 C = tanaka x X far-object 1 chair P (W = C = far-object)p (C = far-object X = x, D = d) x x far-object P (W C, X) P (W = C = tel#, X) P (W = C = staff#, X) 3 interactive alignment 6) lexical alignment 8) P (W C, X) P (X X) P (X = x X = x) x x x x 3 X 1 X 2 x x x P (X = x X = x) 1 P (X = x X = x) REBN P (X 2 X 1, D 2) REBN P (X 2 X 1, D 2 ) P (X 2 D 2 )P (X 2 X 1 ) 4 3. REBN 3.1 one-anaphora one-anaphora 2 P (W C, X) P (W C) 3 P (W C, X) X C X 4 5

6 1 that blue one P (W C, X) P (W = C, X) D(C) P (W = C = chair, X) 2 θ P (W C, X) P (W C, X; θ) REBN X W C REBN C 1 demonstrative demonstrative X 3.1 C 2 21),22) X 3 22) 3 REX-graph REBN W 2 W 1 X 1 X 2 D 2 P (X 2 = x 2 X 1 = x 1 ) x 2 x 1 P (X 2 X 1 ) P (X 3 = x 3 X 2 = x 2) x 2 x 3 18) REBN REX-graph REBN right middle 6

7 4 W1 C1 X1 W2 C2 X2 W3 C3 X3 3 3 (W -X C-D ) REBN 1 REBN D1 D2 D REBN 2 D 1, D 2, D 3 REBN REBN X 3 quantity 3 C 2 quantity 2 REBN 6 W 1 W 2 W 3, W 4 2 P (X 1 X 3 ) P (X 2 X 3 ) 0/ X 1 X 2 X 3 C 1, C 2 2 7

8 W1 C1 X1 W1 C1 W2 C2 W2 C2 X D W3 C3 X2 D 5 W3 C3 3 1 (W -X C-D ) 6 W1 C1 X1 W2 W3 W4 C2 C3 C4 X2 X3 (W -X C-D ) D 7 (W -X C-D ) P (C X, D) C = couple X 2 P (C X, D) 1 D(X) 3.7 another REBN 2 d 2 x 1, x 2 x 1 d x 1 P (C = another X = x 2, D = d) > P (C = another X = x 1, D = d) x REBN 8 24) A,B A B B P (X 1 X 2 ) D(X 1 ) D(X 2 ) 8

9 9 8 Discourse Semantic Parser ASR BN Constructor Situation Ontology Inference Engine Perception Encyclopedic Episodic DB 4. 2 D(X) D(D) (1)REX-graph (2) (3) 9 BN-Constructor Inference Engine ASR Semantic Parser REX-graph BN-Constructor BN- Constructor Discourse Situation Ontology REBN Inference Engine REBN Inference Engine D(X) 1 D(X) else else Inference Engine Perception Situation else 9 D(D) 17) Perception D(X) Situation D(X) D(X) Encyclopedic Episodic DB) Situation 9

10 5. Cho Maida 3) C X C 3.1 primary, secondary, tertiary 3 Roy 14) Roy Cho Maida 24) 3.8 D(X) Salmon-Alt 15) Denis 5) Lison 11) Markov Logic Network (MLN) 13) MLN MLN MLN 6. REX-J 16) 22) McShane 12) 7) 1),10) REBN 1) Asher, N. and Lascarides, A.: Logics of Conversation, Cambridge (2003). 2) Burger, J.D. and Connoly, D.: Probabilistic Resolution of Anaphoric Reference, Proc. AAAI Fall Symposium on Intelligent Probabilistic Approaches to Natural Language (1992). 3) Cho, S. and Maida, A.: Using a Bayesian Framework to Identify the Referent of Definite Descriptions, Proc. AAAI Fall Symposium on Intelligent Probabilistic Approaches to Natural Language (1992). 4) Dale, R. and Reiter, E.: Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions, Cognitive Science, Vol.18, pp (1995). 5) Denis, A.: Generating Referring Expressions with Reference Domain Theory, Proc. the 6th International Natural Language Generation Conference (INLG) (2010). 6) Garrod, S. and Pickering, M.J.: Joint Action, Interactive Alignment, and Dialog, Topics in Cognitive Science, Vol.1, No.2, pp (2009). 7) Grosz, B. and Sidner, C.: Attention, Intentions and the Structure of Discourse, Computational Linguisitics, Vol.12, pp (1986). 8) Janarthanam, S. and Lemon, O.: Learning Lexical Alignment Policies for Generating Referring Expressions for Spoken Dialogue Systems, Proc. the 12th European Workshop on Natural Language Generation (ENLG), pp (2009). 9) Jensen, F.V. and Nielsen, T.D.: Bayesian Networks and Decision Graphs, Springer, second edition (2007). 10

11 10) Kamp, H. and Reyle, U.: From Discourse to Logic, Kluwer Academic Publishers (1993). 11) Lison, P., Ehrler, C. and Kruijff, G.-J.M.: Belief Modelling for Situation Awareness in Human-Robot Interaction, Proc. the 19th International Symposium on Robot and Human Interactive Communication (RO-MAN) (2010). 12) McShane, M.: Reference Resolution Challenges for Intelligenct Agents: The Need for Knowledge, IEEE Intelligent Systems, Vol.24, No.4, pp (2009). 13) Richardson, M. and Domingos, P.: Markov Logic Networks (2006). 14) Roy, D.: Learning Visually-Grounded Words and Syntax for a Scene Description Task, Computer Speech and Language, Vol.16, No.3 (2002). 15) Salmon-Alt, S. and Romary, L.: Generating Referring Expressions in Multimodal Contexts, Proc. the INLG 2000 workshop on Coherence in Generated Multimedia (2000). 16) Spanger, P., Yasuhara, M., Iida, R., Tokunaga, T., Terai, A. and Kuriyama, N.: REX-J: Japanese Referring Expression Corpus of Situated Dialogs, Language Resources and Evaluation (2010). Online First, DOI: /s ) Thórisson, K. R.: Simulated Perceptual Grouping: An Application to Human- Computer Interaction, Proc. the 16th Annual Conference of the Cognitive Science Society, pp (1994). 18) Tokunaga, T., Koyama, T. and Saito, S.: Meaning of Japanese spatial nouns, Proc. the Second ACL-SIGSEM Workshop on The Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications, pp (2005). 19) Weissenbacher, D.: A Bayesian Network for the Resolution of Non-anaphoric Pronoun it, Proc. NIPS 2005 Workshop on Bayesian Methods for Natural Language Processing (2005). 20) Vol.17, No.5, pp (2002). 21) Vol.5, pp.5 16 (2002). 22) Vol.6, No.4, pp (1999). 23) 16 pp (2010). 24) 23 (2009). 11

No. 3 Oct The person to the left of the stool carried the traffic-cone towards the trash-can. α α β α α β α α β α Track2 Track3 Track1 Track0 1

No. 3 Oct The person to the left of the stool carried the traffic-cone towards the trash-can. α α β α α β α α β α Track2 Track3 Track1 Track0 1 ACL2013 TACL 1 ACL2013 Grounded Language Learning from Video Described with Sentences (Yu and Siskind 2013) TACL Transactions of the Association for Computational Linguistics What Makes Writing Great?