UniDic version

Size: px

Start display at page:

Download "UniDic version"

きゅういちとくやす
5 years ago
Views:

1 UniDic version

2 UniDic version Users Manual Yasuharu Den, Atsushi Yamada, Hideki Ogura, Hanae Koiso, and Toshinobu Ogiso Copyright c The UniDic consortium. All rights reserved. version April 2007 version October 2007 version April 2008 version July 2008

3 I UniDic-chasen chasenrc UniDic-mecab dicrc II 11 4 UniDic i

4 A 25 A.1 Version A.2 Version A.3 Version ii

5 UniDic ChaSen UniDic-chasen MeCab UniDicmecabChaSen MeCab ChaSen IPA ipadic UniDic ISTC Tel:

6 I 1 UniDic ChaSenver MeCabver Windows Linux/Cygwin Windows chaunilinux/cygwin Linux/Cygwin configure ChaSen/MeCab./configure --with-use-mecab=0./configure --with-use-chasen=0 # ChaSen # MeCab Cygwin configure Cygwin./configure --with-systemtop=d:/cygwin utf8 ChaSen -i w chasen -i w < UniDic-chasen Windows 1. unidic-chasen139_xxxx.zip unidic-chasen139_xxxx XXXX utf8, sjis, eucj 2. ChaSen C:\Program Files\ChaSen dic dic dic Linux/Cygwin 1. unidic-chasen139_xxxx.tar.gz unidic-chasen139_xxxx 2

7 XXXX utf8, eucj, sjis 2. ChaSen /usr/local/lib/chasen/dic *1 unidic unidic unidic 4. 3 unidic chasenrc $HOME/.chasenrc GRAMMAR Cygwin GRAMMAR Windows (GRAMMAR D:/Cygwin/usr/local/lib/chasen/dic/unidic) UniDic-mecab Windows 1. unidic-mecab139_xxxx.zip unidic-mecab139_xxxx XXXX utf8, sjis, eucj 2. MeCab C:\Program Files\MeCab dic unidic unidic unidic MeCab -d mecab -d "C:\Program Files\MeCab\dic\unidic" Linux/Cygwin 1. unidic-mecab139_xxxx.tar.gz unidic-mecab139_xxxx XXXX utf8, eucj, sjis 2. MeCab /usr/local/lib/mecab/dic *2 unidic unidic unidic MeCab -d mecab -d /usr/local/lib/mecab/dic/unidic 1.3 *1 ChaSen chasen-config --dicdir *2 MeCab mecab-config --dicdir 3

8 1.3.1 UniDic-chasen Windows 1. unidic-chasen139src.zip unidic-chasen139src 2. ChaSen C:\Program Files\ChaSen dic 3. 1 unidic-chasen139src ChaSen 4. 3 unidic-chasen139src Makefile.bat 2 dic 4 Filler.dic.dic utf8 2 Makefile_sjis.bat Makefile_eucj.bat utf8 Linux/Cygwin 1. unidic-chasen139src.tar.gz unidic-chasen139src 2. 1 unidic-chasen139src./configure && make 3. make install /usr/local/lib/chasen/dic/unidic 4. 3 unidic chasenrc $HOME/.chasenrc GRAMMAR 2 configure --with-exclude-dic,./configure --with-exclude-dic=filler.dic Cygwin configure Cygwin./configure --with-systemtop=d:/cygwin GRAMMAR Windows (GRAMMAR D:/Cygwin/usr/local/lib/chasen/dic/unidic) utf8 2 configure with-encoding=sshift-jis eeuc-jp make install utf8 4

9 1.3.2 UniDic-mecab Windows 1. unidic-mecab139src.zip unidic-mecab139src 2. MeCab C:\Program Files\MeCab dic unidic 3. 1 unidic-mecab139src MeCab 4. 3 unidic-mecab139src Makefile.bat 2 unidic 4 Filler.csv.csv utf8 Makefile_sjis.bat Makefile_eucj.bat utf8 MeCab -d mecab -d "C:\Program Files\MeCab\dic\unidic" Linux/Cygwin 1. unidic-mecab139src.tar.gz unidic-mecab139src 2. 1 unidic-mecab139src./configure && make 3. make install /usr/local/lib/mecab/dic/unidic 2 configure --with-exclude-dic,./configure --with-exclude-dic=filler.csv utf8 configure with-charset=sjisshift-jis euc-jpeuc-jp make install utf8 MeCab -d mecab -d /usr/local/lib/mecab/dic/unidic 5

10 2 UniDic-chasen 2.1 grammar.cha % ctypes.cha cforms.cha ( ( () () () () () ()) ( %) ( %)) 2.2 ctypes.cha (( ) ( )) 2.3 cforms.cha (- ((- *) (- *) ( *) ( *) (- *) (- *) (- *) (- *) (- *) (- *) (- *))) 6

11 ChaSen ipadic UniDic-chasen cforms.cha 2.4.dic (POS ( )) ((LEX ( 0)) (READING ) (PRON ) (INFO orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" acontype=" %F2@0, %F1, %F2@-1" goshu="" )) (POS ( )) ((LEX ( 4000)) (READING ) (PRON ) (INFO orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="1" acontype="c3" goshu="" )) amodtype (POS ( )) ((LEX ( 261)) (READING ) (PRON ) (CTYPE -) (CFORM -) (BASE ) (INFO orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="2" acontype="c1" goshu="" )) (POS ( )) ((LEX ( 261)) (READING ) (PRON ) (CTYPE -) (CFORM -) (BASE ) (INFO orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="2" acontype="c1" goshu="" )) (POS ( )) ((LEX ( 261)) (READING ) (PRON ) (CTYPE -) (CFORM ) (BASE ) (INFO orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="2" acontype="c1" amodtype="m2@1" goshu="" ))... (POS ( )) ((LEX ( 261)) (READING ) (PRON ) (CTYPE -) (CFORM -) (BASE ) (INFO orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="2" acontype="c1" amodtype="m4@1" goshu="" )) UniDic-chasen ChaSen INFO 4.4 7

12 2.5 connect.cha (( ((( ))) ((( ))) ) 814) (( ((( ) - -)) ((() - - )) ) 147) (( (((*))) ((() - - )) ) 8000) (( ((( ) * * )) ((() - - )) ) 425) 2.6 chasenrc chasenrc ChaSen (GRAMMAR /usr/local/lib/chasen/dic) (DADIC chadic) (UNKNOWN_POS ( )) (OUTPUT_FORMAT ; 1 "<cha:w1 orth=\"%m\" kana=\"%?u/%m/%y/\" pron=\"%?u/%m/%a/\" pos=\"%u(%p-)\"%?t/ ctype=\"%t \"//%?F/ cform=\"%f \"// %?I/ %i//>%m</cha:w1>\n") (OUTPUT_COMPOUND "SEG") (EOS_STRING "") (DEF_CONN_COST 10000) (POS_COST ((*) 1) ((UNKNOWN) 30000) ) (CONN_WEIGHT 1) (MORPH_WEIGHT 1) (COST_WIDTH 0) (ANNOTATION (("<" ">") "%m\n")) 8

13 GRAMMAR UNKNOWN_POS OUTPUT_FORMAT EOS_STRING ANNOTATION xml < > UniDic-chasen ChaSen xml OUTPUT_FORMAT ; <cha:w1>...</cha:w1> 1 <cha:w1 orth="" kana="" pron="" pos="--" orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="0" acontype="c2" goshu=""></cha:w1> <cha:w1 orth="" kana="" pron="" pos="--" orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="0,1" acontype="c2" goshu=""></cha:w1> <cha:w1 orth="" kana="" pron="" pos="-" orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" acontype=" %F2@0, %F1, %F2@-1" goshu=""></cha:w1> <cha:w1 orth="" kana="" pron="" pos="--" orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="0,1" acontype="c2" goshu=""></cha:w1> <cha:w1 orth="" kana="" pron="" pos="-" ctype="" cform="-" orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" atype="0" acontype="c3" goshu=""></cha:w1> <cha:w1 orth="" kana="" pron="" pos="" ctype="-" cform="-" orthbase="" kanabase="" pronbase="" lform="" lemma="" form="" acontype=" %F4@1" goshu=""></cha:w1> xslt uniutils xml2txt.xsl 9

14 3 UniDic-mecab 3.1.csv UniDic-chasen.dic 3.2.def dicrc dicrc MeCab cost-factor = 700 bos-feature = BOS/EOS,*,*,*,*,*,*,*,*,*,*,*,* eval-size = 9 unk-eval-size = 4 max-grouping-size = 10 output-format-type = unidic node-format-unidic = %m\t%f[10]\t%f[6]\t%f[7]\t%f-[0,1,2,3]\t%f[4]\t%f[5]\t%f[12]\n unk-format-unidic = %m\t%m\t%m\t%m\t%f-[0,1,2,3]\t%f[4]\t%f[5]\t%f[12]\n eos-format-unidic = EOS\n output-format-type node-format-xxx, unk-format-xxx, bos-format-xxx, eos-format-xxx XXX f[0]f[12] MeCab rewrite.def dicrc 10

15 II 4 UniDic UniDic 3 : 22, UniDic , pp , UniDic UniDic UniDic

17 1 1 2 UniDic UniDic chaone chaone 13

18 1 UniDic lform lemma goshu form pos ctype cform itype iform icontype ftype fform fcontype orthbase orth kanabase kana pronbase pron atype amodtype acontype ChaSen chasenrc xml ChaSen LEX, READING, PRON, POS, CTYPE, CFORM INFO 2.4 MeCab 5 UniDic UniDic ipadic IPA - 14

19 {,, }

20 * {,, } {, } - - -{,,,, } *3 16

21 {, } {, }

22 3 18

23 {, } -- - UniDic

24 UniDic

25 6 21

26 7 / 8 N1 N3 / / N4 N6 / N8 / / / Nj Nh Ns Nm Nn :, :, :

27 9 0 1 M1@M N 0 M M2@M N 0 M M 0 M4@M M 0 M 0 M N 0 :, M 0 :, M: 10 C1 N 1 + M 2, C2 N 1 + 1, C3 N 1, C4 0, C5 M 1 N 1 :, M 1 :, M 2 : *4 chaone 3 C2 C1 2 *

28 11 0 N 2 P1 0 N 1 + M 2 P2 N N 1 + M 2 P4 N M 1 P6 0 P13 M 1 P14 M 1 N 1 + M 2 N 1 :, M 1 :, N 2 :, M 2 : 12 0 F1 M 1, F2@M N 1 + M M 1, F3@M M 1 N 1 + M F4@M N 1 + M F5 0 F6@M, L N 1 + M N 1 + L N 1 :, M 1 :, M, L:

29 A A.1 Version A.2 Version MeCab A.3 Version MeCab 25

IPA:セキュアなインターネットサーバー構築に関する調査

IPA:セキュアなインターネットサーバー構築に関する調査 Copyright 2003 IPA, All Rights Reserved. Copyright 2003 IPA, All Rights Reserved. Copyright 2003 IPA, All Rights Reserved. Copyright 2003 IPA, All Rights Reserved. Copyright 2003 IPA, All Rights Reserved.