syslog 1,2,a) 3,b) syslog syslog syslog Interop Tokyo ShowNet syslog Proposal of the anomaly detection method analyzing syslog data using Bollinger Bands algorithm on event network Hiroshi Abe 1,2,a) Mikifumi Shikida 3,b) 1. 1.1 1 2 / 1 IIJ 2 3 a) abe@iij.ad.jp/h-abe@jaist.ac.jp b) shikida.mikifumi@kochi-tech.ac.jp / / syslog syslog 57
1 VMware vrealize LogInsight syslog Interop Tokyo[1] ShowNet[2] syslog 1.2 ShowNet ShowNet Interop Tokyo 2 ShowNet ShowNet ShowNet syslog ShowNet 1.3 ShowNet syslog ShowNet 1 syslog 1 VMware vrealize LogInsight [3](LogInsight) syslog LogInsight, (OSPF down/bgp down/storm detection ) ( ) ShowNet debug info ShowNet syslog 1.4 2, 3 4, 5. 6 7 2. Holt-Winters [4] ShowNet Jon Kleinberg [5] Kleinberg 58
syslog ChangeFinder[6] ChangeFinder. Google word2vec[7]. syslog 3. 3.1 ShowNet syslog ShowNet syslog [8] 3.2 1 1980 John Bollinger 2 2 ( ) 68.26% 2 ( ) 95.44% 3 ( ) 99.73% 95.44% 2 +2-2 + 2 ( ) UpperLimit() - 2 ( ) LowerLimit() () x = 1 n n 1 x i i=0 ( ) σ = 1 n 1 (x i x) 2 n i=0 59
OS CentOS 7.2 1 Python 3.5.1 CPU Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz 128GB { n 1 = 1 n 2 n x 2 i i=0 ( n 1 ) 2 } x i i=0 ( ) 2 UpperLimit, LowerLimit 3.3 syslog syslog 2 ( ) syslog ( 2 ) 95.44% UpperLimit LowerLimit 4.56% 4.56% 4. 4.1 1 ShowNet ShowNet Python syslog 6.4GB 4,350 4.2. syslog. syslog [9] Mmm dd hh:mm:ss IP 1 import pandas as pd 2 df = pd. read_csv (./2016-06 -06 - syslog. log, delim_whitespace =True,...) 3 count = df. groupby (pd. TimeGrouper ( 1 Min )). count () 4 mean = count. rolling ( window =60). mean () 5 std = count. rolling ( window =60). std () 6 std_plus = std. apply ( lambda x: x * 2) 7 std_ minus = std. apply ( lambda x: x * -2) 8 upper_ limit = mean. add ( std_plus ) 9 lower_limit = mean. add ( std_minus ) 3 2 csv csv Python pandas[10] csv pandas DataFrame DataFrame pandas 1 1 DataFrame DataFrame ( : mean : std ) 1 / 1 1 (0 23 ) 1 / 1 60 3 1) pandas 2) (csv ) 3) DataFrame 1 4) mean 5) std 6) 2 (+2 ) 60
4 1 2 Level Low 1-100 Middle 100-1000 High 1000 3 5/27 192 617 34 5.51% 5/28 181,285 1440 111 7.71% 5/29 552,579 1440 96 6.67% 5/30 821,363 1440 88 6.11% 5/31 617,368 1440 71 4.93% 6/1 917,368 1440 82 5.69% 6/2 1,949,738 1440 91 6.32% 6/3 1,771,956 1440 69 4.79% 6/4 2,108,661 1440 75 5.21% 6/5 3,177,122 1440 71 4.93% 6/6 3,297,654 1440 51 3.54% 6/7 2,702,382 1440 67 4.65% 6/8 3,186,363 1440 87 6.04% 6/9 12,769,834 1440 65 4.51% 6/10 9,446,694 1083 65 6.00% 43,500,499 20420 1123 5.50% 7) -2 (-2 ) 8) (UpperLimit) 9) (LowerLimit) 5/27 4 count 1 Upper- Limit LowerLimit UpperLimit LowerLimit +2 UpperLimit UpperLimit +2 2.28% UpperLimit syslog DoS(Denial of Service). +2 ( 2) Low, Middle, High 3 +2 5. 5.1 ShowNet syslog 3 3 ( 1 ) ( 2 ) ( 3 ) ( 4 ) ( 5 ) 5 syslog (1 :60 ) 1 (86400 ) (86400/60=1440) 5/27 1440 1 192 syslog 5/27 syslog. ShowNet syslog. 6/10 1440 17 ShowNet UpperLimit UpperLimit ShowNet 5.50% 94.5% UpperLimit ShowNet (Hotstage) 61
4 Low Middle High 5/27 34 34 0 0 5/28 111 86 25 0 5/29 96 30 64 2 5/30 88 30 42 16 5/31 71 30 30 11 6/1 82 32 38 12 6/2 91 27 39 25 6/3 69 38 23 8 6/4 75 26 38 11 6/5 71 25 39 7 6/6 51 15 22 14 6/7 67 28 36 3 6/8 87 24 57 6 6/9 65 25 39 1 6/10 65 19 35 11 1123 469 527 127 41.76% 46.93% 11.31% 5 6/6 3 Hotstage 5/27 6/3, 6/4 6/7, 6/8 6/10 Hotstage syslog ShowNet. 200 1,200 3% 6% 5.2 2 4 Low 41.76%, Middle 46.93%, High 11.31% Low Middle 88% 5.3 ShowNet High syslog 3 5.3.1 6/6 6/6 18 ( 5) +2 6 6/8 SNMP Get request is recieved. :... SNMP Get response is sent. :... SNMP Get request response 18:10-18:16 ShowNet OID SNMP Get 6/6 18:10-18:16 SNMP SNMP ACL SNMP SNMP Daemon 5.3.2 6/8 6/9 6/9 3 6/1 6/8 100 300 6/9 1,200 6/8 23 6 6/10 1 9,000 6/8 23 6/6 SNMP SNMP 62
7 6/9 94.5% UpperLimit 8 6/10 UpperLimit 6/8 High 6/9( 7) 6/9 1200 4.51% 5.3.3 6/10 8 6/10 8 ShowNet NFV(Network Functional Virtualization) BGP(Border Gateway Protocol) 5 High High 6. 6.1 5.1 ShowNet syslog +2 2.28% 6.2 5.2 Low Middle 88% High High 5.3 3 UpperLimit 6.3 syslog 6.4 ShowNet 5.3 63
2-3 ShowNet 2 1.. 6.5 ShowNet syslog 7. 7.1 ShowNet syslog +2 94.5% UpperLimit. UpperLimit LowerLimit syslog syslog [1] Interop Tokyo, http://www.interop.jp/ [2] ShowNet, http://www.interop.jp/2016/shownet/ [3] VMware vrealize Log Insight, http://www.vmware.com/jp/products/vrealize-loginsight.html [4] Kalekar, Prajakta S.: Time series forecasting using holtwinters exponential smoothing. Kanwal Rekhi School of Information Technology 4329008 (2004): 1-13. [5] Kleinberg,J.: Bursty and Hierarchical Structure in Streams,Proc,8th SIGKDD pp.91101,2002. [6] J. Takeuchi and K. Yamanishi : A Unifying Framework for Detecting Outliers and Change Points from Time Series, IEEE transactions on Knowledge and Data Engineering, vol.18, no.4, pp.482-492, 2006. [7] word2vec, https://github.com/dav/word2vec [8] Bollinger, J. : Bollinger on Bollinger Bands. McGraw Hill, 2002 [9] Gerhards, R : RFC 5424,The Syslog Protocol, March 2009, https://tools.ietf.org/html/rfc5424 [10] pandas, Python Data Analysis Library: http://pandas.pydata.org/ 7.2 ShowNet. ShowNet / 64