1 : Google Amazon Facebook Akamai Hyper giants Web [1] Web Web Web Magnitude Exploit Kit PHP.net Web Yahoo 600 [2] Web URL Blacklist URL Blacklist URL

Size: px

Start display at page:

Download "1 : Google Amazon Facebook Akamai Hyper giants Web [1] Web Web Web Magnitude Exploit Kit PHP.net Web Yahoo 600 [2] Web URL Blacklist URL Blacklist URL"

きみえこびき
5 years ago
Views:

1 Computer Security Symposium October 2014 URL URL NTT {akiyama.mitsuhiro, Web URL URL URL URL URL URL Exploit kit URL URL URL URL Bayesian Sets URL IP Searching URLs that have similar features to the existing malicious URLs Bo Sun Mitsuaki Akiyama Takeshi Yagi Tatsuya Mori Waseda University NTT Secure Platform Laboratories Okubo, Shinjuku-ku, Tokyo Midoricho, Musashino-city, Tokyo {akiyama.mitsuhiro, Abstract URL blacklists/blocklists are widely deployed as an effective means to mitigate various threats targeting web users; e.g., Drive-by-download attacks, phishing scam, etc. Broadly speaking, building a blacklist is a process of picking up malicious URLs to label them. Since there are enormous amount of unlabeled URLs in the wild, it is useful to organize them and make it searchable to the set of existing malicious URLs. This work proposes a novel method that enables such search. An example application of the method is as follows. Given a set of known malicious URLs that make use of a particular exploit kit, the method searches similar URLs to those malicious URLs from a set of unknown URLs. The search results can be sorted with similarity scores. In this work, we adopt the Bayesian sets as a search algorithm that detects similar items to a given set of items. We extract various features such as characterisrtics of strings in URLs, IP address, domain name information etc. We validate and demonstrate the effectiveness of our approach using the real world URLs

2 1 : Google Amazon Facebook Akamai Hyper giants Web [1] Web Web Web Magnitude Exploit Kit PHP.net Web Yahoo 600 [2] Web URL Blacklist URL Blacklist URL URL URL URL Blacklist [3] Web URL Blacklist URL URL URL : URL URL URL URL URL URL URL Web Exploit kit URL URL URL Exploit kit URL URL Bayesian Sets [4] URL IP : URL URL URL Exploit kit URL URL URL 2 URL Choi [5] URL Web DNS DNS Fast Flux Ma [6] URL Eshete [7] URL Web - 2 -

3 Random Tree Xu [8] URL Correlation feature selection (CFS) Canali Prophiler [9] URL URL URL HTML Javascript URL J48 Ma [10] ground truth URL URL Invernizzi [11] Canali Evilseed Canali Invernizzi Google Bing Yacy URL Google s Safe Browsing Wepawet URL 5 SEO DNS URL URL EvilSeed 3 URL URL Bayesian Sets Bayesian Sets URL 3.1 Bayesian sets Bayesian Sets [4] Google Sets [12] Ghahramani Google Sets 1 : Toyota Nissan Honda Google Sets BMW Ford Audi Mitsubishi Mazda Volkswagen... Ghahramani Google Sets Bayesian Sets Bayesian Sets D URL x D URL URL Q D Q Q x S S(x; Q) = P (x, Q) P (x)p (Q) = P (x Q) P (x) Bayesian Sets Q x D x 1 Google Sets Google Sheets - 3 -

4 i URL x i = {x i1,..., x im } m x ij {0, 1} (1 j m) θ j : P (x ij θ j ) = θ x ij j (1 θ j ) 1 x ij S(x i ; Q) = P (x i Q) P (xi θ)p (θ Q)dθ = P (x i ) P (xi θ)p (θ)dθ θ B(α, β) α, β [4] S(x i ; Q) = P (x i Q, α, β) P (x i α, β) m ( ) ( xij α j + β j αj βj = α j + β j + N j=1 N = Q α j = α j + x i Q x ij α j β j = β j + (1 x ij ) x i Q β j ) 1 xij log S(x i ; Q) α, β x ij m j = x i D x ij/ D α j = cm j β j = c(1 m j ) α j /(α j +β j ) m j [4] c = 2 URL D α, β Q α, β 3.2 URL URL URL URL FQDN IP whois URL 1 Bayesian Sets 1: URL No. 1 URL URL URL exe 1 11 IP 1 12 IP URL No. 1 3 URL D p (p = 0, 10, 20,..., 100) 10 URL x 93.4 URL 11 {1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1} 1 x 1 x No. 4 6 No. 7 9 & %? / = -. 8 No. 12 IP No. 1 3 Ma [6] - 4 -

5 No Xu [8] Canali [9] IP URL No. 12 2: URL URL URL URL 823 URL URL 2,495 URL 6,636 URL 1,674 URL URL 16, : URL URL URL ALEXA 30,000 DMOZ 20, URL URL URL 2 URL, URL URL. URL URL Web URL URL URL URL Web Marionette [13] BotnetWatcher [14] URL Phishtank [15] URL URL 2013 URL 1, URL 4.2 URL URL Alexa [16] DMOZ [17] Web URL FQDN FQDN URL 3 URL URL (Exploit) URL Q = N = 3 3 URL URL 5.1 Bayesian sets - 5 -

The number of URLs with Exploit;MS06-014 1800 1600 1400 1200 1000 800 600 400 200 Queries with same URL and IP feature Queries with same URL feature (1474,965) (1474,542) 0 1: 2: URL URL IP (U ) URL

6 The number of URLs with Exploit;MS Queries with same URL and IP feature Queries with same URL feature (1474,965) (1474,542) 0 1: 2: URL URL IP (U ) URL IP (UI ) URL IP (I ) IP URL I URL URL (Exploit MS06-014) U UI 1 URL URL UI U IP Exploit Kit URL URL U UI UI URL URL U URL UI 5.2 URL URL 3 URL URL 4 Exploit Kit URL Expoloit Kit URL URL FQDN A IP IP Exploit Kit URL URL 2 URL URL URL ,000 URL 2-6 -

7 4: URL URL IP URL(Exploit;MS06-014) ** 2011/12/20 URL(Exploit;MS06-014) ** 2011/12/17 URL(Exploit;MS06-014) ** 2011/12/17 IP URL(Exploit;MS06-014) ** 2011/12/17 URL(Exploit;MS06-014) ** 2011/12/17 URL(Exploit;MS06-014) ** 2011/12/20 URL(Exploit;MS06-014) ** 2012/1/19 URL(Exploit;MS06-014) ** 2011/12/31 URL(Exploit;MS06-014) ** 2011/12/17 URL(Exploit;MS06-014) ** 2011/12/17 URL(Exploit;MS06-014) ** 2012/9/13 URL(Exploit;MS06-014) ** 2012/5/12 URL(Exploit;MS06-014) ** 2011/12/13 URL(Exploit;MS06-014) ** 2012/4/26 URL(Exploit;MS06-014) ** 2012/9/25 URL(Exploit;MS06-014) ** 2012/3/8 URL(Exploit;MS06-014) ** 2011/12/27 URL(Exploit;MS06-014) ** 2011/12/17 Fraction of Exploit; MS URLs(%) Fraction of Malware download URLs (%) : URL 4: URL 3 URL URL % 1,000 70% URL % 1,000 70% URL URL 5 URL 20 URL 8 URL URL IP URL 6 3 URL URL URL 10,000 95% URL URL URL URL - 7 -

8 Fraction of phishing URLs (%) Fraction of generic malicious URLs (%) : URL Query1: Exploit Query2: Malware Download Query3: Phishing : URL URL 6 Bayesian Sets URL URL URL URL URL URL URL [1] C. Labovitz, S. Iekel-Johnson, D. McPherson, J. Oberheide, and F. Jahanian, Internet inter-domain traffic, SIGCOMM Comput. Commun. Rev., vol. 41, pp., Aug [2] Steve Ragan, An in-depth look at one of the Web s most famous crime kits. com/article/ /malware-cybercrime/ exposed-an-inside-look-at-the-magnitude-exploit-kit. html. [3] D. Canali, M. Cova, G. Vigna, and C. Kruegel, Prophiler: A fast filter for the large-scale detection of malicious web pages, in Proc. WWW, (New York, NY, USA), pp , ACM, [4] Z. Ghahramani and K. A. Heller, Bayesian sets, in Proc. NIPS, [5] H. Choi, B. B. Zhu, and H. Lee, Detecting malicious web links and identifying their attack types, in Proc. USENIX WebApps, [6] J. Ma, L. K. Saul, S. Savage, and G. M. Voelker, Beyond blacklists: learning to detect malicious web sites from suspicious urls, in Proc. KDD, pp , [7] B. Eshete, A. Villafiorita, and K. Weldemariam, Binspect: Holistic analysis and detection of malicious web pages, in Proc. SecureComm, pp , [8] L. Xu, Z. Zhan, S. Xu, and K. Ye, Cross-layer detection of malicious websites, in Proc. CODASPY, pp , [9] D. Canali, M. Cova, G. Vigna, and C. Kruegel, Prophiler: a fast filter for the large-scale detection of malicious web pages, in Proc. WWW, pp , [10] J. Ma, L. K. Saul, S. Savage, and G. M. Voelker, Identifying suspicious urls: an application of large-scale online learning, in Proc. ICML, p. 86, [11] L. Invernizzi and P. M. Comparetti, Evilseed: A guided approach to finding malicious web pages, in Proc. IEEE Symposium on Security and Privacy, pp , [12] Google Sets. of_google_products#discontinued_in_2011. [13] M. Akiyama, M. Iwamura, Y. Kawakoya, K. Aoki, and M. Itoh, Design and implementation of high interaction client honeypot for drive-by-download attacks, IEICE Transactions, vol. 93-B, no. 5, pp , [14] K. Aoki, T. Yagi, M. Iwamura, and M. Itoh, Controlling malware http communications in dynamic analysis system using search engine, in Proc. IEEE CSS, pp. 1 6, [15] PHISHTANK, Free community site for anti-phishing service. [16] ALEXA, The web information company. [17] DMOZ, Netscape open directory project

Landing Landing Intermediate Exploit Exploit Distribution Provos [1] Drive-by Download (Exploit Distribution ) Drive-by Download (FCDBD: Framework for

Landing Landing Intermediate Exploit Exploit Distribution Provos [1] Drive-by Download (Exploit Distribution ) Drive-by Download (FCDBD: Framework for Drive-by Download Web 1,a) 1,b) 1,c) Web Web Web Drive-by Download FCDBD(Framework for Countering Drive-By Download) FCDBD Drive-by Download Landing Web Landing Web JavaScript Web Drive-by Download