( )

Size: px

Start display at page:

Download "( )"

ありさみやまる
5 years ago
Views:

1 NAIST-IS-MT

2 ( )

3 SNS SNS 3 3 Chantokun, NAIST-IS- MT , i

4 Automatic Error Correction for Learners of Japanese as a Second Language Seiji Kasahara Abstract Although the number of Japanese learners is increasing, there are not enough Japanese teachers who can teach Japanese as a second language. Hence language learning web services such as language learning SNS are getting popular. In this thesis we improve learning environment of Japanese as second language in three different aspects. The first is error correcting romaji-kana conversion. This conversion system helps editors correct Japanese learners composition. Our system outperformed traditional input methods. The second is case-marker correction, which occupies a large part of Japanese learners errors. Our system uses a noisy channel model, which has been used in machine translation, to reflect tendencies of Japanese learners errors. Experimental results show that it achieved 4.7% higher accuracy than the one without the error model of Japanese learners. The third is a user interface for Japanese learning support system. We constructed a system which is easy for learners to use. Three attempts complementarily support Japanese education more effectively. Keywords: Japanese education, Japanese error correction, case-marker correction, learners corpus, Romaji-kana conversion, spelling correction, Web interface, Chantokun Master s Thesis, Department of Information Processing, Graduate School of Information Science, Nara Institute of Science and Technology, NAIST-IS-MT , February 2, ii

5 Lang Lang iii

6 Noisy Channel Model Noisy Channel Model Chantokun A. 52 A iv

7 55 v

8 1 Lang Noisy Channel Model λ Lang Lang NAIST vi

9 10 Lang NAIST JSON ( ) a b vii

10 Web 2 3 iknow! 4 SNS Livemocha 5 Lang

11 1.2 3 SNS SNS 2

12 3 3

13 Web ,601 3, [13] 7 1. E

14 8 DB DB DB NAIST DB [15] DB 10 NAIST NAIST Lang-8 SNS Lang-8 10 [3] Lang ,

15 1 Lang-8 925, % 763,971 10,000 1 Lang-8 Lang OK desu hanasemasu hanashimasu mada made 9 ha no 10 amerikajin americagen amerika america jin gen 6

16 1 Lang-8 1 Onaka ga itai desu! Onaka ga itai desu! 2 suki ni narimasu. suki ni narimasu.perfect! 3 Isogashikatta. Isogashikatta. 4 gakko wa omoshiroi desu. gakko wa omoshiroi desu. 5 Tokyo ni irutoki, Tokyo ni irutoki, Meiji-jingu mo ni ikimashita. Meiji-jingu ni mo ikimashita. 6 Noh ni mimashita. Nihonjin no tomodachi ga Noh wo misetekuremashita. 7 Konnichiwa! OK desu 8 nihongo ga sukoshi hanashimasu nihongo ga sukoshi hanasemasu demo made jouzu ja arimasen. demo mada jouzu ja arimasen. 9 Chichi no atama ga ii desu. Chichi ha atama ga ii desu. 10 watashi wa americagen desu. watashi wa amerikajin desu. HTML NAIST NAIST DB DB 7

17 NAIST DB 2 4 8

18 2 9

19 3. 10%

20 SNS SNS Lang [14] N [9, 1] 11

21 [2] [3] Lang-8 SNS Lang-8 12 Lang-8 ha wa wo o he e 11 A

22 %(320,655/849,894) [3] 22.0%(186,807/849,894) 155,287 WordNet IPADic CaboCha , kakasi

23 IPADic 2 Step Step Step1 1 Step Step Step1: 17 N N 1 packu

24 2 Lang-8 yorushiku onegia shimasu. yoroshiku onegai shimasu. Muscle musical wo mietai. Muscle musical wo mitai. Muscle musical Gorofu ga daisuki desu gorufu ga daisuki desu 163 kau pakku chikau [4] 18 Step2: N kakasi SRILM Witten-Bell ca, ci, cu, ce, co ka, shi, ku, se, ko m n

25 kinyuu n 3.6 Lang = N t N e, = N t N w N t N w N e Anthy Anthy

26 3 Anthy Lang-8 Lang % Anthy 74.5% % % 78.6% 76.6% 17

27 4 shuutmatsu t shuumatsu do-yoobi doyoubi packu c pakku 5 1 Soshite, kurama wo durivu wo shimasu, Soshite, kuruma wo doraibu wo shimasu, 2 boku wa nagai ichi-nichi no renshou o shimasu boku wa nagai ichi nichi no renshuu o shimasu 3 Terebi gamu wo asobitai desu terebi geemu wo asobitai desu 78.1%

28 renshuu renshou renshou N muzukashii musugashi doumo domou 5 6 su shi [17] s sh 7 n ng ng s sh 8 Holland 9 19

29 [8] nouryokushiken nouryoku shiken 2 IPADic 2 nouryokushiken

30 6 1 domou doumo 2 Yorushiko onegai shimasu yoroshiku onegai shimasu 3 Merrii kurisamasu, mina-san merii kurisumasu minasan 4 domo arigato guzaimasu doumo arigatou gozaimasu 5 nihongo ga scoshi wakarimasu 6 hajimimashtei s nihongo ga sukoshi wakarimasu sh hajimemashite 7 donna eigaosaiking mimashitaka 8 Horandajin desu g donna eiga wo saikin mimashitaka orandajin desu 9 Nihon go wa totemo musugashi desu nihon go wa totemo muzukashii desu 21

31 7 durivu doraibu 3 prutugarogo p porutogarugo 3 musugashi muzukashii 3 8 denwabango denwa bangou Meiji-jingu meiji jinguu noryoukushiken nouryoku shiken 22

32 NAIST % SNS 7.6% NTT [12] [18] SVM [5] 23

33 [16] 1 [7] Rozovskaya [6] [3]

34 [10] 21 NAIST 3, % 0.06% 0.0% 13.93% % 25

35 9 NAIST % % % % % % % % % % % % % % % % % % % % 26

36 and or 0.71% kakujoshilist = (,,,,,,,,,,,,,,,,, ) (1) 4.4 Noisy channel model w n 1 = w 1, w 2, w 3,..., w n P (w n 1 ) P (w i w i 1 1 ) N 23 27

37 N N 1 N P (w 1 w n 1 1 ) = P (w n w n 1 n N+1) N = 1, 2, 3 w1 n 2 n P (w1 n ) = P (w i w i 1, w i 2 ) (2) i Web N 24 ŵ 2 = arg max P (w 2 w 1, w 3 ) = arg max C(w 1, w 2, w 3 ) w 2 w 2 P (w 2 w 1, w 3 ) w 1, w 2, w 3 C(w 1, w 2, w 3 ) w 2 kakujoshilist 1 Backoff N 2011 Web N Backoff N 0 N-1 Backoff

38 Algorithm 1 Backoff correct arg max w 2 P (w 2 w 1, w 3 ) if correct == NONE then correct arg max w 2 (P (w 2 w 1 )P (w 2 w 3 )) end if if correct == NONE then correct arg max w 2 P (w 2 ) end if Noisy Channel Model Noisy channel model ŵ C = arg max w C P (w C w E ) w E w C P (w C w E ) P (w E w C ) P (w C ) P (w E ) w E kakujoshilist 1 P (w C w E ) ŵ C P (w C w E ) P (w C w E ) = P (w E w C )P (w C ) P (w E ) 29

39 arg max P (w C w E ) = arg max P (w E w C )P (w C ) w C w C P (w E w C ) P (w C ) Backoff Noisy Channel Model arg max P (w C w E ) = arg max(p (w E w C )P (w C )) w C w C arg max w C P (w C w E ) = arg max w C (log P (w E w C ) + log P (w C )) λ(0.0 λ ) arg max w C (λ log P (w E w C ) + log P (w C )) λ λ 25 3 λ = ,

40 3 Noisy Channel Model λ λ = 1.6 λ = Noisy channel model Web N 1 Lang-8 NAIST 31

41 Web N 1 Web N , Web N 7 32

42 10 Lang ,239 18,991 17,906 13,446 12,959 12,748 11,740 10,831 8,105 8, P (w 2 w 1, w 3 ) = C(w 1, w 2, w 3 ) w 2 C(w 1, w 2, w 3 ) w 2 kakujoshilist 1 SNS Lang-8 [3] [11] 1 33

43 11 NAIST 368 (11.0%) 920 (27.2%) 2,093 (61.9%) 328 (9.7%) 920 (27.2%) 1,787 (52.9%) kakujoshilist 328 (9.7%) 812 (24%) 1,485 (43.9%) kakujoshilist P (w E w C ) = C(w E, w C ) w E C(w E, w C ) w E, w C kakujoshilist NAIST NAIST 6, NAIST 80% 34

44 NAIST kakujoshilist N IPADic MeCab Noisy channel model NAIST Web N Noisy channel model 6.4%

45 % 58.3% 31.8% Noisy Channel Model 54.4% 66.1% 31.8% Noisy Channel Model 55.6% 67.5% 32.1% 13 5 Noisy channel model

13 (Backoff ) Noisy Channel Model NoisyChannelModel 75 60% 95 75% 99 79% 126 64 70% 65 71% 63 68% 92 38 76% 30 60% 28 56% 50 70 65% 82 76% 84 78% 108 0 0% 0 0% 0 0% 9 5

46 13 (Backoff ) Noisy Channel Model NoisyChannelModel 75 60% 95 75% 99 79% % 65 71% 63 68% % 30 60% 28 56% % 82 76% 84 78% % 0 0% 0 0% % 3 38% 3 38% % 35 55% 35 55% % 0 0% 0 0% 2 0 0% 0 0% 0 0% % 71 71% 77 77% % 0 0% 0 0% 5 0 0% 0 0% 0 0% % 0 0% 0 0% 9 0 0% 0 0% 0 0%

47 14 1 [16] Web N N Web N 28 Web Web N N Noisy channel model

48 N 39

49 rikaichan 30 rikaichan ROBO-SENSEI

6 31 32 HTML 5.2 Chantokun N 33 Chantokun 34 31 http://usf.usfca.

50 HTML 5.2 Chantokun N 33 Chantokun

51 Firefox (3.6.10) Google Chrome (v ) Internet Explorer

52 2011 EDICT 35 MySQL Twitter 36 Twitter HTML Flash

9 10 HTML CSS JavaScript Ajax Ajax JavaScript jquery 5.2.2 IPADic-2.7.0 37 MeCab-0.

53 9 10 HTML CSS JavaScript Ajax Ajax JavaScript jquery IPADic MeCab kakujoshilist kakujoshilist = (,,,,,,,,,,,,,,,,, ) (3)

54 kakujoshilist w 1, w 2, w 3 4 w 2 kakujoshilist P (w 2 w 1, w 3 ) = C(w 1, w 2, w 3 ) w 2 C(w 1, w 2, w 3 ) (4) 45

55 w 1, w 3 w 2 joshilist ŵ 2 ŵ 2 = arg max P (w 2 w 1, w 3 ) = arg max C(w 1, w 2, w 3 ) w 2 w CGI 12 JSON JSON 46

56 15 JSON node id int word string error string float correct string score 39 float JSON { items :[ { node id :0, word :, error : None :0, correct : None, score : None }, { node id :1, word :, error : None :0, correct : None, score : None }, { node id :2, word :, error : None :0, correct : None, score : None }, { node id :3, word :, error : kakujoshi : , correct :, score : }, { node id :4, word :, error : None :0, correct : None, score : None } ]} 15 4 Web N ssgnc ssgnc Key-Value Store Kyoto Cabinet error score error

57 4 Kyoto Cabinet kakujoshilist 18 ssgnc Kyoto Cabinet w 2 C(w 1, w 2, w 3 ) Kyoto Cabinet 1 C(w 1, w 2, w 3 ) ssgnc Chantokun API 48

59 Chantokun PM NAIST 50

60 Lang-8 Noisy channel model Lang-8 51

61 A A A

62 16 1 ( ) a i u e o ka ki ku ke ko kya kyu kyo sa si su se so sya syu syo ta ti tu te to tya tyu tyo na ni nu ne no nya nyu nyo ha hi hu he ho hya hyu hyo ma mi mu me mo mya my myo ya (i) yu (e) yo ra ri ru re ro rya ryu ryo wa (i) (u) (e) (o) ga gi gu ge go gya gyu gyo za zi zu ze zo zya zyu zyo da (zi) (zu) de do (zya) (zyu) (zyo) ba bi bu be bo bya byu byo pa pi pu pe po pya pyu pyo a sha shi shu sho tsu cha chi chu cho fu ja ji ju jo b di du dya dyu dyo kwa gwa wo 53

63 1. n 2. n y n

64 [1] Zheng Chen and Kai-Fu Lee. A New Statistical Approach to Chinese Pinyin Input. In Proceedings of the Association for Computational Linguistics (ACL), pp , [2] Yo Ehara and Kumiko Tanaka-Ishii. Multilingual Text Entry using Automatic Language Detection. In Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP), pp , [3] Tomoya Mizumoto, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. Mining Revision Log of Language Learning SNS for Automated Japanese Error Correction of Second Language Learners. In Proceedings of the International Joint Conference on Natural Language Processing (IJC- NLP), pp , [4] Naoaki Okazaki and Jun ichi Tsujii. Simple and Efficient Algorithm for Approximate Dictionary Matching. In Proceedings of the International Conference on Computational Linguistics (COLING), pp , [5] Hiromi Oyama. Automatic Error Detection Method for Japanese Particles. ICT for Analysis, Learning, and Teaching of Languages (ICTATLL), pp , [6] Alla Rozovskaya and Dan Roth. Generating Confusion Sets for Contextsensitive Error Correction. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp , [7] Hisami Suzuki and Kristina Toutanova. Learning to Predict Case Markers in Japanese. In Proceedings of the Association for Computational Linguistics (ACL), pp , [8] Kumiko Tanaka-Ishii, Yusuke Inutsuka, and Masato Takeichi. Japanese Input System With Digits Can Japanese be input only with consonants? In 55

65 Proceedings of the Human Language Technologies (HLT), pp , [9] Yabin Zheng, Chen Li, and Maosong Sun. CHIME: An Efficient Error- Tolerant Chinese Pinyin Input Method. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp , [10],.., [11],,,,.. 18, [12],,,,..., No. 13, pp , [13] (1990).. PDF, [14]. N-gram., Vol. 40, No. 6, pp , [15].., Vol. 54, pp , [16],,,,.. 17, pp , [17],,,,,.., [18],,. ( ).., No. 94, pp ,

ル札幌市公式ホームページガイドライン

ル札幌市公式ホームページガイドライン平成 16 年 1 月 8 日総 ) 広報部長決裁企 ) 情報化推進部長決裁最近改正平成 23 年 3 月 10 日 ...3...3...4...5...5...5...5...6...6...7...8...9...9...10...11...11...12...12...13...13...14...15...15...16...17...18...19...20