Elastic stack Jun Ohtani 1

Size: px

Start display at page:

Download "Elastic stack Jun Ohtani 1"

あいとたにしき
5 years ago
Views:

1 Elastic stack Jun Ohtani 1

info Elasticsearch, founded in 2012 Products: Elasticsearch, Logstash,

2 about Me, Jun Ohtani / Technical Advocate lucene-gosen ElasticSearch Server Elasticsearch, founded in 2012 Products: Elasticsearch, Logstash, Kibana, Beats X-Pack, Elastic Cloud, Professional services: Support & development subscriptions Trainings, Consulting, SaaS 2

3 Elastic Stack 3

4 Wikipedia 4

5 5

6 Logs Logs Logs, many devices, many systems More than 40% of our customers use our products for operational log analysis 6

7 Web 1.2TB 7

8 Elastic Stack 8

9 Beats Elastic Cloud Elsaticsearch Libbeat: beats API Logstash 30 beats 9

10 FILEBEAT METRICBEAT PACKETBEAT WINGLOGBEAT Window 30 Beats Apachebeat, dockbeat, httpbeat, mysqlbeat, nginxbeat, redis beats, twitterbeat, and more 10

11 Metricbeat Collect system and application metrics

12 Metricbeat lots of modules

Beats <3 containerization Monitor your Docker and Kubernetes deployments with ease New Kubernetes module in Metricbeat CPU, memory, bytes on network and more.

13 Beats <3 containerization Monitor your Docker and Kubernetes deployments with ease New Kubernetes module in Metricbeat CPU, memory, bytes on network and more. New processor to add_docker_metadata Container ID, name, image, labels New processor to add_kubernetes_metadata Pod name, pod namespace, container name, pod labels 13

14 Filebeat tail log from file

15 Filebeat many modules

16 Packetbeat Capture the Packet

17 Packetbeat Capture the Packet

18 winlogbeat Welcome to 1998

19 winlogbeat Now

20 Logstash

21 Elasticsearch Heart of the Elastic Stack 21

22 Kibana Window into the Elastic Stack Elastic Stack Apps 22

23 100, M+ 3,700+ Statistics since 2012, founding of Elastic 23

24 130 Millions of Downloads 40 Cumulative downloads of the Elastic Stack (Elasticsearch, Kibana, Beats, Logstash) and X-Pack

25 Demo

26 Horizontal Scale Real-Time Data Availability Flexible Data Model Rapid Query Execution Sophisticated Query Language Schemaless 26

27 Tech Finance Telco Consumer 27

28 Security Alerting X-Pack Monitoring Reporting Graph Machine Learning 28

29 Elastic Cloud Available in AWS today Available in Google Cloud Platform (soon) Available as a private cloud/on-premise solution (Elastic Cloud Enterprise) 29

30 Elastic Cloud Enterprise 30

31 Elastic Stack X-Pack Elastic Cloud Application Search Metrics Analytics Log Analytics Business Analytics Security Analytics Many more 31

32 Search and analytics, it all started here More than 60% of our customers have a search or analytics use case 32

33 33

34 Logs Logs Logs, many devices, many systems More than 40% of our customers use our products for operational log analysis 34

36 Sniff sniff sniff, find the bad actors in your data 200% YoY growth in security use cases with our products 36

37 37 We mine and analyze 4 billion events every day to detect security hacks and threats.

38 75% of our customers use our products for multiple use cases LOG ANALYTICS METRICS SECURITY SEARCH OPERATIONAL ANALYTICS CUSTOM APPS 38

39 39 1,000+ developers use the Elastic Stack for use cases from trade tracking to creating new HR and compliance apps.

41 IT Operational Analytics Security Analytics Business Analytics Spiked 404 errors Unusual DNS activity Rare log messages Web attack Data exfiltration Failing sensor 41

42 Use Case Operational Analytics?? Error?

43 Use Case Security Analytics?? DNS?

44 Use Case Telemetry / Sensors ISP???

45 Where s the anomaly? Visual inspection is not practical 45

46 What s the right threshold? Rule-based alerts are insufficient 46

47 X-Pack 47

48 Discuss Web Elastic{ON}

Thanks for listening! Q & A We re hiring! https://www.elastic.

49 Thanks for listening! Q & A We re hiring! We re helping!

ETL Webinar

ETL Webinar Logstash Filebeat ETL February 23rd, 2018 Kosho Owa, Principal Solution Architect Jun Ohtani, Developer Evangelist The Elastic Stack RESTFul Elasticsearch Elastic Stack Filebeat Beats The Beats family