IPSJ-TOD

Similar documents
1 1 tf-idf tf-idf i

3807 (3)(2) ,267 1 Fig. 1 Advertisement to the author of a blog. 3 (1) (2) (3) (2) (1) TV 2-0 Adsense (2) Web ) 6) 3

08医療情報学22_1_水流final.PDF

kut-paper-template.dvi

( )

一般社団法人電子情報通信学会 THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGIN

Vol. 9 No. 5 Oct (?,?) A B C D 132

A Japanese Word Dependency Corpus ÆüËܸì¤Îñ¸ì·¸¤ê¼õ¤±¥³¡¼¥Ñ¥¹

2

Vol.54 No (July 2013) [9] [10] [11] [12], [13] 1 Fig. 1 Flowchart of the proposed system. c 2013 Information

main.dvi

IPSJ SIG Technical Report Vol.2011-EC-19 No /3/ ,.,., Peg-Scope Viewer,,.,,,,. Utilization of Watching Logs for Support of Multi-

21 Pitman-Yor Pitman- Yor [7] n -gram W w n-gram G Pitman-Yor P Y (d, θ, G 0 ) (1) G P Y (d, θ, G 0 ) (1) Pitman-Yor d, θ, G 0 d 0 d 1 θ Pitman-Yor G

untitled

1 Fig. 1 Extraction of motion,.,,, 4,,, 3., 1, 2. 2.,. CHLAC,. 2.1,. (256 ).,., CHLAC. CHLAC, HLAC. 2.3 (HLAC ) r,.,. HLAC. N. 2 HLAC Fig. 2

[4], [5] [6] [7] [7], [8] [9] 70 [3] 85 40% [10] Snowdon 50 [5] Kemper [3] 2.2 [11], [12], [13] [14] [15] [16]

1 7.35% 74.0% linefeed point c 200 Information Processing Society of Japan

Q [4] 2. [3] [5] ϵ- Q Q CO CO [4] Q Q [1] i = X ln n i + C (1) n i i n n i i i n i = n X i i C exploration exploitation [4] Q Q Q ϵ 1 ϵ 3. [3] [5] [4]

1 Web [2] Web [3] [4] [5], [6] [7] [8] S.W. [9] 3. MeetingShelf Web MeetingShelf MeetingShelf (1) (2) (3) (4) (5) Web MeetingShelf

IPSJ SIG Technical Report Vol.2011-MUS-91 No /7/ , 3 1 Design and Implementation on a System for Learning Songs by Presenting Musical St

skeiji.final.dvi

2. Twitter Twitter 2.1 Twitter Twitter( ) Twitter Twitter ( 1 ) RT ReTweet RT ReTweet RT ( 2 ) URL Twitter Twitter 140 URL URL URL 140 URL URL

gengo.dvi

Computational Semantics 1 category specificity Warrington (1975); Warrington & Shallice (1979, 1984) 2 basic level superiority 3 super-ordinate catego

01-望月.indd

1: A/B/C/D Fig. 1 Modeling Based on Difference in Agitation Method artisoc[7] A D 2017 Information Processing

No. 3 Oct The person to the left of the stool carried the traffic-cone towards the trash-can. α α β α α β α α β α Track2 Track3 Track1 Track0 1

& Vol.5 No (Oct. 2015) TV 1,2,a) , Augmented TV TV AR Augmented Reality 3DCG TV Estimation of TV Screen Position and Ro

2007/2 Vol. J90 D No Web 2. 1 [3] [2], [11] [18] [14] YELLOW [16] [8] tfidf [19] 2. 2 / 30% 90% [24] 2. 3 [4], [21] 428

/4 2012

The copyright of this material is retained by the Information Processing Society of Japan (IPSJ). The material has been made available on the website

¥ì¥·¥Ô¤Î¸À¸ì½èÍý¤Î¸½¾õ

Vol. 42 No MUC-6 6) 90% 2) MUC-6 MET-1 7),8) 7 90% 1 MUC IREX-NE 9) 10),11) 1) MUCMET 12) IREX-NE 13) ARPA 1987 MUC 1992 TREC IREX-N

IPSJ SIG Technical Report Vol.2009-HCI-134 No /7/17 1. RDB Wiki Wiki RDB SQL Wiki Wiki RDB Wiki RDB Wiki A Wiki System Enhanced by Visibl

Mimehand II[1] [2] 1 Suzuki [3] [3] [4] (1) (2) 1 [5] (3) 50 (4) 指文字, 3% (25 個 ) 漢字手話 + 指文字, 10% (80 個 ) 漢字手話, 43% (357 個 ) 地名 漢字手話 + 指文字, 21

第62巻 第1号 平成24年4月/石こうを用いた木材ペレット

( ) [1] [4] ( ) 2. [5] [6] Piano Tutor[7] [1], [2], [8], [9] Radiobaton[10] Two Finger Piano[11] Coloring-in Piano[12] ism[13] MIDI MIDI 1 Fig. 1 Syst

日本感性工学会論文誌

DEIM Forum 2010 A Web Abstract Classification Method for Revie

IPSJ SIG Technical Report Pitman-Yor 1 1 Pitman-Yor n-gram A proposal of the melody generation method using hierarchical pitman-yor language model Aki

1 Fogg Fogg Behavior Model [1] information cascade [2] TPO [3] Fig. 2 Target area of this paper. 1 Fig. 1 Fogg b

Rapp BLEU[10] [9] BLEU OrthoBLEU Rapp OrthoBLEU [9] OrthoBLEU OrthoBLEU ) ) ) 1) NTT Natural Language Research

IPSJ SIG Technical Report Vol.2012-CG-148 No /8/29 3DCG 1,a) On rigid body animation taking into account the 3D computer graphics came

HASC2012corpus HASC Challenge 2010,2011 HASC2011corpus( 116, 4898), HASC2012corpus( 136, 7668) HASC2012corpus HASC2012corpus

Unknown

2006 [3] Scratch Squeak PEN [4] PenFlowchart 2 3 PenFlowchart 4 PenFlowchart PEN xdncl PEN [5] PEN xdncl DNCL 1 1 [6] 1 PEN Fig. 1 The PEN

IPSJ SIG Technical Report Vol.2010-NL-199 No /11/ treebank ( ) KWIC /MeCab / Morphological and Dependency Structure Annotated Corp

DPA,, ShareLog 3) 4) 2.2 Strino Strino STRain-based user Interface with tacticle of elastic Natural ObjectsStrino 1 Strino ) PC Log-Log (2007 6)

1 4 4 [3] SNS 5 SNS , ,000 [2] c 2013 Information Processing Society of Japan

IPSJ SIG Technical Report Secret Tap Secret Tap Secret Flick 1 An Examination of Icon-based User Authentication Method Using Flick Input for

main.dvi

1 1 CodeDrummer CodeMusician CodeDrummer Fig. 1 Overview of proposal system c

Vol.55 No (Jan. 2014) saccess 6 saccess 7 saccess 2. [3] p.33 * B (A) (B) (C) (D) (E) (F) *1 [3], [4] Web PDF a m

IPSJ SIG Technical Report Vol.2009-DPS-141 No.20 Vol.2009-GN-73 No.20 Vol.2009-EIP-46 No /11/27 1. MIERUKEN 1 2 MIERUKEN MIERUKEN MIERUKEN: Spe

自然言語処理21_249

Vol. 42 No. SIG 8(TOD 10) July HTML 100 Development of Authoring and Delivery System for Synchronized Contents and Experiment on High Spe

258 5) GPS 1 GPS 6) GPS DP 7) 8) 10) GPS GPS ) GPS Global Positioning System

IPSJ SIG Technical Report Vol.2014-CG-155 No /6/28 1,a) 1,2,3 1 3,4 CG An Interpolation Method of Different Flow Fields using Polar Inter

IPSJ SIG Technical Report Vol.2009-BIO-17 No /5/26 DNA 1 1 DNA DNA DNA DNA Correcting read errors on DNA sequences determined by Pyrosequencing

_314I01BM浅谷2.indd

Microsoft Excelを用いた分子軌道の描画の実習

IPSJ SIG Technical Report Vol.2014-HCI-158 No /5/22 1,a) 2 2 3,b) Development of visualization technique expressing rainfall changing conditions

Kyushu Communication Studies 第2号

独立行政法人情報通信研究機構 Development of the Information Analysis System WISDOM KIDAWARA Yutaka NICT Knowledge Clustered Group researched and developed the infor

TF-IDF TDF-IDF TDF-IDF Extracting Impression of Sightseeing Spots from Blogs for Supporting Selection of Spots to Visit in Travel Sat

[2] , [3] 2. 2 [4] 2. 3 BABOK BABOK(Business Analysis Body of Knowledge) BABOK IIBA(International Institute of Business Analysis) BABOK 7

udc-3.dvi

H19国際学研究科_02.indd

IPSJ SIG Technical Report Vol.2015-SE-187 No /3/ Checking the Consisteny between Requirements Specification Documents and Regulations A

58 10

The 18th Game Programming Workshop ,a) 1,b) 1,c) 2,d) 1,e) 1,f) Adapting One-Player Mahjong Players to Four-Player Mahjong

[2] OCR [3], [4] [5] [6] [4], [7] [8], [9] 1 [10] Fig. 1 Current arrangement and size of ruby. 2 Fig. 2 Typography combined with printing

Studies of Foot Form for Footwear Design (Part 9) : Characteristics of the Foot Form of Young and Elder Women Based on their Sizes of Ball Joint Girth

IPSJ SIG Technical Report Vol.2013-GN-86 No.35 Vol.2013-CDS-6 No /1/17 1,a) 2,b) (1) (2) (3) Development of Mobile Multilingual Medical

Modal Phrase MP because but 2 IP Inflection Phrase IP as long as if IP 3 VP Verb Phrase VP while before [ MP MP [ IP IP [ VP VP ]]] [ MP [ IP [ VP ]]]

28 TCG SURF Card recognition using SURF in TCG play video

Table 1 Table 2

Izard 10 [1]Plutchik 8 [2] [3] Izard Neviarouskaya [4][5] 2.2 Hao [6] 1 Twitter[a] a) Shook Wikipedia

: ( 1) () 1. ( 1) 2. ( 1) 3. ( 2)

IPSJ SIG Technical Report Vol.2014-NL-219 No /12/17 1,a) Graham Neubig 1,b) Sakriani Sakti 1,c) 1,d) 1,e) 1. [23] 1(a) 1(b) [19] n-best [1] 1 N

Input image Initialize variables Loop for period of oscillation Update height map Make shade image Change property of image Output image Change time L


untitled

(a) 1 (b) 3. Gilbert Pernicka[2] Treibitz Schechner[3] Narasimhan [4] Kim [5] Nayar [6] [7][8][9] 2. X X X [10] [11] L L t L s L = L t + L s


Sobel Canny i

Core1 FabScalar VerilogHDL Cache Cache FabScalar 1 CoreConnect[2] Wishbone[3] AMBA[4] AMBA 1 AMBA ARM L2 AMBA2.0 AMBA2.0 FabScalar AHB APB AHB AMBA2.0

Vol. 23 No. 4 Oct Kitchen of the Future 1 Kitchen of the Future 1 1 Kitchen of the Future LCD [7], [8] (Kitchen of the Future ) WWW [7], [3

<8ED089EF8B D312D30914F95742E696E6464>

[1] B =b 1 b n P (S B) S S O = {o 1,2, o 1,3,, o 1,n, o 2,3,, o i,j,, o n 1,n } D = {d 1, d 2,, d n 1 } S = O, D o i,j 1 i

IPSJ SIG Technical Report Vol.2017-SLP-115 No /2/18 1,a) 1 1,2 Sakriani Sakti [1][2] [3][4] [5][6][7] [8] [9] 1 Nara Institute of Scie

大学における原価計算教育の現状と課題

% 95% 2002, 2004, Dunkel 1986, p.100 1

DT pdf

J. JILA 64 (5), 2001

Abstract

Stepwise Chow Test * Chow Test Chow Test Stepwise Chow Test Stepwise Chow Test Stepwise Chow Test Riddell Riddell first step second step sub-step Step

IT,, i

ICT a) Caption Presentation Method with Speech Expression Utilizing Speech Bubble Shapes for Video Content Yuko KONYA a) and Itiro SIIO 1. Graduate Sc

4.1 % 7.5 %

Transcription:

Vol. 3 No. 2 91 101 (June 2010) 1 1 1 2 1 TSC2 Automatic Evaluation of Text Summaries by Using Paraphrase Kazuho Hirahara, 1 Hidetsugu Nanba, 1 Toshiyuki Takezawa 1 and Manabu Okumura 2 The evaluation of computer-produced summaries has been recognized as an important research problem for automatic text summarization. Traditionally, computer-produced summaries were evaluated automatically by n-gram overlap with human-produced texts. However, these methods cannot evaluate summaries correctly, if the n-grams do not overlap between computer-produced and human-produced summaries, even though the two summaries convey the same meaning. We explored the use of paraphrases for the refinement of traditional automatic methods for summary evaluation. To confirm the effectiveness of our method, we conducted some experiments using the data from the Text Summarization Challenge 2. We found that the use of paraphrases created using a statistical machine translation technique improved the traditional evaluation methods. 1. 13) 4),5) 18) 3 4 5 6 7 1 Graduate School of Information Sciences, Hiroshima City University 2 Precision and Intelligence Laboratory, Tokyo Institute of Technology 91 c 2010 Information Processing Society of Japan

92 2. 2.1 2.2 2.1 BLEU ROUGE BLEU 14) BLEU 1 N 9) BLEU BLEU ROUGE 10) Lin ROUGE 1 ROUGE-N ROUGE-N N S {Reference Summaries} gram ROUGE-N = Count n S match(gram n ) Count(gram (1) S {Reference Summaries} gram n S n) n N gram n Count match (gram n ) N Lin N 1 4 N =1 2 N=1 2.2 6) 16) 1) 16) 2 carbon dioxide carbon dioxide 1 1) 8),11) 2 2 7),18) Zhou ParaEval 18) (1) (2) 2 1 2 Greedy 3 1 2 ROUGE Zhou ParaEval ROUGE Zhou 3 1 2 Zhou ParaEval Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

93 Kauchak 7) Kauchak 3. TSC2 3) 30 10 20% 10 100 1 318 318 5 A 64 20.1% B 78 24.5% C 38 11.9% 1 5 1 10 100 D 39 12.3% E 99 31.2% A 3 B C D X Y X Y E A D Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

94 4. 4.1 4.2 A D 4.1 2 ParaEval ParaEval ParaEval ParaEval ParaEval (1) C D (2) (1) B A (3) (1) (2) (4) (1) (2) (3) (1) (2) Greedy (4) Lin 10) ParaEval ParaEval (1) (2) (1) C D (3) (1) (2) B A (4) (1) (2) (3) 4.2 ParaEval 4 SMT SMT DS WN WordNet NTT NTT SMT Zhou X Y X Y The Daily Yomiuri 150,000 17) GIZA++ 1 2 85,858 3 DS 1) (1) CaboCha 3 56 1 http://www.fjoch.com/giza++.html 2 3 http://chasen.org/ taku/software/cabocha/ Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

95 1 Table 1 Paraphrasing methods for text evaluation. Table 2 2 Correspondence of the classification of paraphrases and paraphrasing methods. SMT DS WordNet WN NTT NTT (2) (1) (3) (4) SMART 15) (2) (3) WordNet WN WordNet 2009 3 2), 1 WordNet synset WordNet synset NTT NTT NTT 4 1 2 3 4 4 1 http://nlpwww.nict.go.jp/wn-ja/ SMT DS Word NTT Net A 20.1% (64/318) B 24.5% (78/318) C 11.9% (38/318) D 12.3% (39/318) E 31.2% (99/318) 5. 4 5.1 TSC2 3) 1,150 30 20 20% 600 20 10 10 2 600 3 1 10 10 8 1 1 2 Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

96 3 3 4 Table 3 The distribution of evaluation results by three human subjects. Table 4 List of 15 proposed methods and a baseline method. A B C 4 0 0 29 5 53 0 67 6 89 38 180 7 143 372 200 8 165 170 93 9 89 20 27 10 61 0 4 1 3 10 A D 4 2, 3 SMT DS NTT NTT WordNet WN 4 15 ROUGE-1 4 4 5 EX-1 1 3 A 7.55 B 7.29 C 6.58 3 2 3 10 1 4 3 + A B C D 5.5 1 3 6 4 B 6.5 5.5 3 4 D 4.5 3 1 4 2.1ROUGE Lin 10) ROUGE-1 ROUGE-2 TSC2 Nanba 12) ROUGE-1 ROUGE-1 5 2 1 1 SMT DS WordNet NTT S D W N SD SW SN DW DN WN SDW SDN SWN DWN SDWN ROUGE-1 EX-4 4.1 ParaEval EX-5 EX-8 ParaEval ParaEval EX-1 9 EX-2 9 EX-3 9 EX-4 9 ParaEval EX-5 9 EX-6 9 Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

97 EX-7 9 EX-8 9 3 3 1.5 39 4 1 30 1 4 5.2 ParaEval ParaEval 5 6 30 5 6 5 EX-1 ROUGE-1 5 15 DW ROUGE-1 0.027 8.1% 6 15 DN ROUGE-1 0.020 5.9% 1 9 B 4 9 5 4 4 Table 5 Table 6 5 ParaEval Evaluation results using an extract-type reference summary (ParaEval method). EX-1 EX-2 S SMT 0.280 0.326 D DS 0.338 0.379 W WordNet 0.332 0.376 N NTT 0.332 0.367 SD 0.340 0.369 SW 0.358 0.336 SN 0.276 0.338 DW 0.359 0.326 DN 0.343 0.374 WN 0.332 0.376 SDW 0.339 0.331 SDN 0.348 0.356 SWN 0.346 0.350 DWN 0.358 0.327 SDWN 0.340 0.326 ROUGE-1 0.332 0.376 6 ParaEval Evaluation results using an abstract-type reference summary (ParaEval method). EX-3 EX-4 S SMT 0.334 0.364 D DS 0.349 0.421 W WordNet 0.337 0.448 N NTT 0.337 0.428 SD 0.348 0.374 SW 0.337 0.435 SN 0.294 0.352 DW 0.334 0.420 DN 0.357 0.403 WN 0.337 0.448 SDW 0.325 0.412 SDN 0.345 0.374 SWN 0.341 0.424 DWN 0.326 0.416 SDWN 0.329 0.400 ROUGE-1 0.337 0.448 Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

98 7 ParaEval 8 ParaEval Table 7 Evaluation results using an extract-type reference summary (Reverse ParaEval method). Table 8 Evaluation results using an abstract-type reference summary (Reverse ParaEval method). EX-5 EX-6 EX-7 EX-8 S SMT 0.265 0.373 D DS 0.377 0.409 W WordNet 0.346 0.398 N NTT 0.350 0.398 SD 0.343 0.390 SW 0.337 0.382 SN 0.270 0.384 DW 0.348 0.381 DN 0.373 0.409 WN 0.346 0.398 SDW 0.340 0.380 SDN 0.335 0.389 SWN 0.342 0.383 DWN 0.345 0.383 SDWN 0.334 0.382 S SMT 0.308 0.352 D DS 0.337 0.420 W WordNet 0.336 0.440 N NTT 0.335 0.437 SD 0.347 0.389 SW 0.349 0.377 SN 0.310 0.349 DW 0.349 0.375 DN 0.339 0.424 WN 0.336 0.440 SDW 0.350 0.380 SDN 0.342 0.394 SWN 0.368 0.383 DWN 0.351 0.367 SDWN 0.359 0.373 ROUGE-1 0.332 0.376 ROUGE-1 0.337 0.448 ParaEval ParaEval 7 8 7 8 7 D ROUGE-1 0.045 13.6% 8 SWN ROUGE-1 0.031 9.2% ParaEval 6. EX-1 ROUGE-1 EX-3 EX-1 EX-4 1 ParaEval ParaEval EX-1 EX-4 ParaEval EX-5 EX-8 ParaEval EX-2 EX-6 4.1 ParaEval ParaEval ParaEval (1) ParaEval (2) Para- Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

99 Eval (2) ParaEval (3) 1 ParaEval ParaEval ParaEval EX-1 EX-4 ParaEval EX-5 EX-8 SDWN 1 ParaEval 5.2 49.1 ParaEval 0 4.6 ParaEval ParaEval ParaEval ParaEval 4.2 4 SMT DS ParaEval 1 ParaEval EX-1 EX-5 EX-3 EX-7 EX-8 EX-4 1 Kauchak 7) 2 place Kauchak ParaEval 4 D W Word- Net S N NTT S N 30 Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

100 7. Zhou ParaEval Zhou WordNet NTT TSC2 0.045 ParaEval ParaEval 1) Vol.49, No.3, pp.1426 1436 (2008). 2) Bond, F., Isahara, H., Uchimoto, K., Kuribayashi, T. and Kanzaki, K: Extending the Japanese WordNet 15 pp.80 83 (2009). 3) Fukushima, T., Okumura, M. and Nanba, H: Text Summarization Challenge 2/Text Summarization Evaluation at NTCIR Workshop 3, Working Notes of the 3rd NTCIR Workshop Meeting, PART V, pp.1 7 (2002). 4) Vol.47, No.6, pp.1753 1766 (2006). 5) Hovy, E., Lin, C.-Y., Zhou, L. and Fukumoto, J: Automated Summarization Evaluation with Basic Elements, Proc. 5th Conference on Language Resources and Evaluation (2006). 6) Vol.11, No.5, pp.151 198 (2004) 7) Kauchak, D. and Barzilay, R: Paraphrasing for Automatic Evaluation, Proc. 2006 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL), pp.455 462 (2006). 8) Lee, L: Measures of Distributional Similarity, Proc. 37th Annual Meeting of the Association for Computational Linguistics, pp.25 32 (1999). 9) Lin, C.-Y. and Hovy, E: Automatic Evaluation of Summaries Using N-gram Cooccurrence Statistics, Proc. 4th Meeting of the North American Chapter of the Association for Computational Linguistics and Human Language Technology, pp.150 157 (2003). 10) Lin, C.-Y.: ROUGE: A Package for Automatic Evaluation of Summaries, Proc. ACL-04 Workshop Text Summarization Branches Out, pp.74 81 (2004). 11) Lin, D.: Automatic Retrieval and Clustering of Similar Words, Proc. 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics, pp.768 774 (1998). 12) Nanba, H. and Okumura, M: An Automatic Method for Summary Evaluation Using Multiple Evaluation Results by a Manual Method, Proc. COLING/ACL 2006 Main Conference Poster Sessions, pp.603 610 (2006). 13) Vol.23, No.1, pp.10 16 (2008). 14) Papineni, K., Roukos, S., Ward, T. and Zhu, W.-J.: BLEU: A Method for Automatic Evaluation of Machine Translation, IBM Research Report, RC22176 (W0109-0220) (2001). 15) Salton, G.: The SMART Retrieval System. Experiments in Automatic Document Processing, Prentice-Hall, Inc., Upper Saddle River, NJ (1971). 16) 14 pp.123 126 (2008). 17) Utiyama, M. and Isahara, H.: Reliable Measures for Aligning Japanese-English News Articles and Sentences, Proc. 41st Annual Meeting of the Association for Computational Linguistics, pp.72 79 (2003). 18) Zhou, L., Lin, C.-Y., Munteanu, D.S. and Hovy, E.: ParaEval: Using Paraphrases to Evaluate Summaries Automatically, Proc. 2006 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL), pp.447 454 (2006). ( 21 12 20 ) ( 22 4 7 ) Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan

101 2008 2010 1984 1989 2007 18 ISS 1996 1998 2001 2002 2010 ACL ACM 1962 1984 1989 1992 2000 2009 AAAI ACL Vol. 3 No. 2 91 101 (June 2010) c 2010 Information Processing Society of Japan