24 SPAM Performance Comparison of Machine Learning Algorithms for SPAM Discrimination

Size: px

Start display at page:

Download "24 SPAM Performance Comparison of Machine Learning Algorithms for SPAM Discrimination"

かずただたけはな
5 years ago
Views:

1 24 SPAM Performance Comparison of Machine Learning Algorithms for SPAM Discrimination

2 SPAM SPAM SPAM SPAM SVM AdaBoost RandomForest SPAM SPAM UCI Machine Learning Repository Spambase SPAM SPAM SPAM Bayes % SPAM Bayes 42.8 % % SPAM AdaBoost Random Forest i

3 Abstract Performance Comparison of Machine Learning Algorithms for SPAM Discrimination Fujimori Natsuki Many other machine learning techniques are able to applied for classification, and quantitative comparison is required. In this research, Naive Bayes Classifier, Neural Network, Support Vector Machine (SVM), Bagging, AdaBoost, and Random Forest are applied to classify written in both English and Japanese in order to filter SPAM mail out. Those algorithms are compared with each other from a viewpoint of classification precision. For English classification, the dataset Spambase of UCI Machine Learning Repository is used. Total number of the data is 4601, and training data is from 500 to 4000, which are randomly selected, and the rest are the test data. For Japanese classification, original corpus is created and used. Total number of the data is 1400 and training data are randomly selected to SPAM in English discrimination is performed under the same conditions to compare the result of precision. As a result, for English SPAM distinction, all algorithms except Naive Bayes Classifier achieves the precision exceeding 90 %. Moreover, for Japanese SPAM, Naive Bayes Classifier became a distinction rate of 42.8 %, and % abtained by other algorithms. key words SPAM, Machine Learning, Naive Bayes Classifier, Neural Network, Support Vector Machine, Bagging, AdaBoost, Random Forest ii

4 (Bagging) AdaBoost Random Forest SPAM SPAM SPAM SPAM SVM iii

5 SVM SPAM A 29 iv

6 AdaBoost Random Forest SPAM SVM 8 SPAM SVM 8 SPAM SPAM SPAM SPAM A A A v

7 SPAM SPAM vi

8 1 SPAM [1] SPAM SPAM SPAM SPAM SPAM SPAM SPAM SPAM SPAM 6 SPAM 1

9 Bayes NN SVM AdaBoost Random Forest RF Hewlett-Packard Labs Mark Hopkins UCI Machine Learning Repository Spambase Data Set SPAM SPAM SPAM HAM 1400 TF IDF SPAM Bayes 5 90 SPAM Bayes SPAM 4 SPAM 2

10 2 SPAM Naive Bayes classifier P (B A) = P (A B)P (B) P (A) (2.1) P (A), P (B) A B P (B A) A B 100 SPAM SPAM 60 HAM B S SPAM B H HAM P (B S ) SPAM P (B H ) HAM P (B S ) = 40/100 = 0.4 P (B H ) = 60/100 = 0.6 SPAM P (A), P (A B) A = P (A = ) = (5+11)/100 = 0.16 P (A = B S ) SPAM 3

11 2.1 term SPAM HAM P (A = B S ) = 5/40 = P (A = B H ) = 11/60 = SPAM SPAM HAM P (B) P (A) 0.16 P (A B) HAM SPAM P (SP AM ) /0.16 = HAM P (HAM ) /0.16 = P (SP AM ) > P (HAM ) HAM Mozilla Thunderbird 4

12 入力層中間層出力層 x wih xi i h whk k xi... I H K 2.1 Neural Network David E. Rumelhart [3] y k = ϕ 0 (α k + Σ h w hk ϕ h (α h + Σ i w ih x i )) (2.2) ϕ α 5

13 w(j + 1) = w(j) + η δ R (2.3) w(j + 1) j + 1 w(j) j η δ R x1 マージンサポートベクター 0 x

14 2.4 (Bagging) Support Vector Machine, SVM Vladimir N.Vapnik 1992 SVM [2] SVM 2.4 (Bagging) B 個のブーストストラップに分割学習データ弱学習機多数決結果を出力 2.3 Bagging Leo Breiman

15 2.5 AdaBoost bootstrap n 2. m 3. h B B {h i i = 1, 2,..., B} H(x) = arg max {i h i = y} 2.5 AdaBoost 学習データ分割訓練データテストデータ重み wi 弱学習機 1 誤判別率信頼度 α 重み更新弱学習機 2 誤判別率信頼度 α 重み更新多数決弱学習機 T 誤判別率信頼度 α 重み更新強学習機 2.4 AdaBoost 8

16 2.6 Random Forest AdaBoost Yoav Freund Robert Schapire 1996 AdaBoost 1/ AdaBoost AdaBoost 1. N 2. T 3. w ti = 1/N 4. t h α 7. w (h+1)i T 9. α 2.6 Random Forest Random Forest Leo Breiman Random Forest 9

17 2.7 学習データランダムサンプリング B 個のブーストストラップに分割弱学習機多数決結果を出力 2.5 Random Forest Random Forest Random Forest 1. B Cross Validation

18 2.7 学習データ分割 ⅰ 評価データ訓練データ ⅱ 評価データ訓練データ ⅷ 訓練データ 8 評価データ 1. a a

19 3 SPAM OS CPU 3.1 Windows 7 Enterprise Intel(R) Core(TM) i5-2400s 2.50GHz 4.00GB R x MeCab 3.2 SPAM SPAM UCI Machine Learning Repository Spambase Data Set 4601 Spam or NonSpam SPAM 2788 HAM

20 3.3 SPAM 3.3 SPAM SPAM データ形態素解析 (RMeCab) 分かち書き (.csv file) 1.txt 2.txt 3.txt DF 総和各単語が全文書中に出現した回数 (DF 値 ) および各単語の出現頻度総和を算出上位 10% をコーパスの特徴量として抽出 SPAM

21 3.3 SPAM 4. Document Frequency, DF DF 10 % % SPAM HAM type

22 3.3 SPAM サイト無料様今回

23 3.3 SPAM SPAM 600 HAM

24 SPAM Bayes NN SVM AdaBoost RF SPAM 17

25 4.1 SPAM 判別率 Bayes NN SVM バギング AdaBoost RF 訓練データ数 4.1 SPAM NN RF Random Forest

26 4.1 SPAM SVM ガウシアン線形多項式タンジェントラプラシアンベッセル ANOVA スプライン SVM 8 SPAM 判別率訓練データ数ガウシアン線形多項式タンジェントラプラシアンベッセル ANOVA スプライン 4.3 SVM 8 SPAM 4.2 SVM

27 4.1 SPAM % 6 Random Forest SVM RBF % Random Forest % 96.3 % SVM % AdaBoost % 94.3 % 80 % 20

28 SPAM Bayes NN SVM Bagging AdaBoost RF SPAM 判別率 Bayes NN SVM Bagging AdaBoost RF 訓練データ数 SPAM 21

29 4.2 SPAM SVM 50% SPAM 8 ANOVA 6 SPAM 90% % R 90% SPAM SPAM 4.6 NN RF Random Forest

30 4.2 SPAM Bayes NN SVM AdaBoost RF SPAM % SVM 23

4.2 SPAM 判別率 1 0.9 0.8 0.7 0.6 0.5 0.4 0.

31 4.2 SPAM 判別率英文日本語文機械学習手法名 SPAM 24

32 5 SPAM SPAM 6 University of California, Irbine Machine Learning Repository 500 Random Forest SVM 8 ANOVA SPAM 1000 SVM SPAM SPAM 25

33 5 Free BSD 4 SNS 2 26

35 [1], 45, [2] Nello Cristianini, John Shawe-Taylor,,,p.9,. [3], R,p251,. 28

36 A 判別率 NN SVM バギング AdaBoost RF 訓練データ数 A

37 判別率訓練データ数ガウシアン線形多項式タンジェント ANOVA A 判別率 NN SVM Bagging AdaBoost RF 訓練データ数 A

..,,,, , ( ) 3.,., 3.,., 500, 233.,, 3,,.,, i

..,,,, , ( ) 3.,., 3.,., 500, 233.,, 3,,.,, i 25 Feature Selection for Prediction of Stock Price Time Series 1140357 2014 2 28 ..,,,,. 2013 1 1 12 31, ( ) 3.,., 3.,., 500, 233.,, 3,,.,, i Abstract Feature Selection for Prediction of Stock Price Time