NTT daichi@cslab.kecl.ntt.co.jp IPSJ SIGNL 190 2009-3-25 ( )
? 99% 99%
: / (Jin, 2006) MDL ( 2007) etc.. / ( 1996( ); Goldwater+ 2006)
n - n NPYLM: Nested Pitman-Yor Language Model Byproduct HPYLM( n-gram )
: HPYLM n-gram Pitman-Yor (PY) : PY: Kneser-Ney HPYLM [Teh 2006] HPYLM = Kneser-Ney n
HPYLM: PY : V PY n-gram= HPYLM ME ( )
NPYLM: - HPYLM HPYLM-HPYLM Markov HPYLM, 1 ( 1/6879)
NPYLM : : : MCMC.
Gibbs Sampling X Z p(x,z) p(z_i X)
Blocked Gibbs Sampler for NPYLM.... : 0. For s=s_1 s_x do parse_trivial(s, ). 1. For j = 1..M do For s=randperm(s_1 s_x) do words(s) words(s) p(w s, ) words(s) done.
Gibbs Sampling 1 2 10 50 100 200
words(s) p(w s, ) : s Forward-Backward (Viterbi ) Forward : t k t-k t-k+1 t Y X Y k Y : j
= k EOS k
( ) t-k-1-j-1-i t-k-1-j-1 t-k-1 t Forward : t k j ;_; ( )
: Goldwater+(ACL 2006) HDP CHILDES WAtsDIs WAts DIs (What s this) : Precision, Recall, F 9,790 9.8 /
10 55 700 600 500 400 300 200 100 0 17 HDP NPYLM HDP(Goldwater+ ACL 2006): 1 2 ( ) NPYLM: 3 -
: SIGHAN Bakeoff 2005 4 : 37,400 : 1000 ( ) : MSR, : CITYU : 50,000 : 2
(F ) NPY(2),NPY(3) NPYLM or + NPY( ) NPY(3) 2 : ZK08 (Zhao&Kit 2008) ZK08
NPYLM
Arabic Gigawords 40,000 (Arabic AFP news) الفلسطينيبسببتظاهرةلانصارحرآةالمقاومةالاسلاميةحماس واذاتحققذلكفانكيسلوفسكييكونقدحازثلاثجواي زآبرىفيابرزثلاثة المحليةوالدوليةللحصولعلىلوازمهمالصحية Google translate: لايتمتعبلقب+ري يس+بلهو+قاي د Filstinebsbptazahrplansarhrkpalmquaompalaslam مايسمىب+السلطةالفلسطينية اعلنتشرطةجنوبافريقيااليومالاثنينانمالايقل iphamas. وقالتدانييلتومسونالتيكتبتالسيناريو وقداستغرقاعدادهخمسةاعوام تاريخي NPYLM الفلسطيني بسبب تظاهرة ل انصار حرآة المقاومة الاسلامية حماس و اذا تحقق ذلك ف ان آيسلوفسكي يكون قد حاز ثلاث جواي ز آبرىفيابرز ثلاثة المحليةو الدولية translate: للحصولعلى لوازم Google هم الصحية لايتمتعof ب لقب + because ري يس + event ب ل the هو + of قاي د + supporters مايسمى ب + السلطة Palestinian الفلسطينية Hamas. اعلن ت Movement, شرطة Resistance جنوبافريقي ا اليوم the Islamic الاثنين انمالايقل وقد استغرق اعداد ه خمسةاعوام. و قال ت دان ييل تومسون التي " تاريخي
Alice in Wonderland first,shedreamedoflittlealiceherself,andonceagainthetinyhandswereclaspedup onherknee,andthebrighteagereyeswerelookingupintohersshecouldhearthevery tonesofhervoice,andseethatqueerlittletossofherheadtokeepbackthewanderingh airthatwouldalwaysgetintohereyesandstillasshelistened,orseemedtolisten,thew holeplacearoundherbecamealivethestrangecreaturesofherlittlesister'sdream.the longgrassrustledatherfeetasthewhiterabbithurriedbythefrightenedmousesplashe dhiswaythroughtheneighbouringpoolshecouldheartherattleoftheteacupsasthema rchhareandhisfriendssharedtheirneverendingmeal,andtheshrillvoiceofthequeen first, she dream ed of little alice herself,and once again the tiny hand s were clasped upon her knee,and the bright eager eyes were looking up into hers -- shecould hearthe very tone s of her voice, and see that queer little toss of herhead to keep back the wandering hair that would always get into hereyes -- and still as she listened, or seemed to listen, thewhole place a round her became alive the strange creatures of her little sister 'sdream. thelong grass rustled ather feet as thewhitera bbit hurried by -- the frightened mouse splashed his way through the neighbour ing pool -- shecould hearthe rattle ofthe tea cups as the marchhare and his friends shared their never -endingme a l,and the
:.
n - n MCMC Kneser-Ney n
HDP-LDA Gibbs (Welling+, NIPS 2007-2008) Loglinear CRF Forward-Backward POS Tagging: CRF+HMM (, + 2007)