1 2014 4 7
lumeta internet mapping http://www.lumeta.com http://www.cheswick.com/ches/map/ 2 / 41
( ) 3 / 41
(Internet measurement and data analysis) : <kjc@sfc.keio.ac.jp> TA: <sora@sfc.wide.ad.jp> SA: TBA URL: http://web.sfc.keio.ac.jp/~kjc/classes/ sfc2014s-measurement/ ( TA SA ): <imda2014s@sfc.wide.ad.jp> : : 2 (20% ) (60%) 4 / 41
( ) ( ) 5 / 41
最近 Big Data が騒がれている 6 / 41
big data big data: : 7 / 41
8 / 41
e- : ( ) ( ) 9 / 41
: WIDE 2001 1570 10 / 41
ITS 3 (, ) 11 / 41
Google s Chief Economist Hal Varian on Statistics The McKinsey Quarterly, January 2009 I keep saying the sexy job in the next ten years will be statisticians. People think I m joking, but who would ve guessed that computer engineers would ve been the sexy job of the 1990s? The ability to take data to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it that s going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for elementary school kids, for high school kids, for college kids. Because now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it. 12 / 41
CPU 13 / 41
14 / 41
15 / 41
: 1973 UC source:http://mathemathinking.blogspot.jp/2012/06/simpsons-paradox.html 16 / 41
(Kenjiro Cho) (2010-) (2002-) WIDE (2001-) 1984 ( ) OS 1993 1996 ( ) (QoS ) 2001 ( ) 2004 ( ) 17 / 41
18 / 41
(1/5) 1 (4/7) : ruby 2 (4/14) ( ) : gnuplot 3 (4/21) : 19 / 41
(2/5) 4 (4/28) : 1 5 (5/12) Web : 6 (5/19) : 20 / 41
(3/5) 7 (5/26) (geo-location) : 8 (6/2) : 2 9 (6/9) : 21 / 41
(4/5) 10 (6/16) : 11 (6/23) : 12 (6/30) : PageRank 22 / 41
(5/5) 13 (7/7) MapReduce : MapReduce 14 (7/14) 23 / 41
( ) 24 / 41
25 / 41
26 / 41
: 27 / 41
(metric) 28 / 41
29 / 41
30 / 41
90 web 2000 P2P SNS 30 dst address 25 20 Traffic (Mbps) 15 10 5 0 00:00:00 04/12 03:00:00 04/12 06:00:00 04/12 09:00:00 04/12 12:00:00 04/12 15:00:00 04/12 18:00:00 04/12 21:00:00 04/12 00:00:00 04/13 Time total 0.0.0.0/0 148.65.7.36 167.210.0.0/17 160.0.0.0/5 202.0.0.0/8 135.0.0.0/10 148.65.0.0/16 128.0.0.0/5 167.208.0.0/12 192.0.0.0/4 129.13.28.0/17 135.43.0.0/17 167.215.33.42 129.13.0.0/17 202.0.0.0/7 31 / 41
PC 32 / 41
( ) (PageRank) SPAM Web (MapReduce) SNS 33 / 41
34 / 41
Ruby
Ruby 1993 : Ruby on Rails (Web ) Ruby Ruby official site: http://www.ruby-lang.org/ Ruby : http://www.ruby-lang.org/ja/documentation/ Ruby : http://jp.rubyist.net/magazine/?firststepruby 36 / 41
Ruby : : ( ) : / Ruby : 37 / 41
Ruby commands irb: Ruby $ irb --simple-prompt >> puts "Hello" Hello ruby: Ruby $ ruby test.rb $ ruby -e puts "Hello".reverse olleh 38 / 41
: filename = ARGV[0] count = 0 file = open(filename) while text = file.gets count += 1 end file.close puts count count.rb $ ruby count.rb foo.txt Ruby #!/usr/bin/env ruby count = 0 ARGF.each_line do line count += 1 end puts count 39 / 41
[1] Ruby official site. http://www.ruby-lang.org/ [2] gnuplot official site. http://gnuplot.info/ [3] Mark Crovella and Balachander Krishnamurthy. Internet measurement: infrastructure, traffic, and applications. Wiley, 2006. [4] Pang-Ning Tan, Michael Steinbach and Vipin Kumar. Introduction to Data Mining. Addison Wesley, 2006. [5] Raj Jain. The art of computer systems performance analysis. Wiley, 1991. [6] Toby Segaran. ( )... 2008. [7] Chris Sanders. ( ). 2 Wireshark.. 2012. [8]... 2011. [9],.., 2010. [10],.., 2009. 40 / 41
2 (4/14) ( ) : gnuplot 41 / 41