Elastic stack Jun Ohtani 2017/12/06 @johtani 1
about Me, Jun Ohtani / Technical Advocate lucene-gosen ElasticSearch Server http://blog.johtani.info Elasticsearch, founded in 2012 Products: Elasticsearch, Logstash, Kibana, Beats X-Pack, Elastic Cloud, Professional services: Support & development subscriptions Trainings, Consulting, SaaS 2
Elastic Stack 3
Wikipedia 4
5
Logs Logs Logs, many devices, many systems More than 40% of our customers use our products for operational log analysis 6
Web 1.2TB 7
Elastic Stack 8
Beats Elastic Cloud Elsaticsearch Libbeat: beats API Logstash 30 beats 9
FILEBEAT METRICBEAT PACKETBEAT WINGLOGBEAT Window 30 Beats Apachebeat, dockbeat, httpbeat, mysqlbeat, nginxbeat, redis beats, twitterbeat, and more 10
Metricbeat Collect system and application metrics
Metricbeat lots of modules
Beats <3 containerization Monitor your Docker and Kubernetes deployments with ease New Kubernetes module in Metricbeat CPU, memory, bytes on network and more. New processor to add_docker_metadata Container ID, name, image, labels New processor to add_kubernetes_metadata Pod name, pod namespace, container name, pod labels 13
Filebeat tail log from file
Filebeat many modules
Packetbeat Capture the Packet
Packetbeat Capture the Packet
winlogbeat Welcome to 1998
winlogbeat Now
Logstash 200 20
Elasticsearch Heart of the Elastic Stack 21
Kibana Window into the Elastic Stack Elastic Stack Apps 22
100,000+ 130M+ 3,700+ Statistics since 2012, founding of Elastic 23
130 Millions of Downloads 40 Cumulative downloads of the Elastic Stack (Elasticsearch, Kibana, Beats, Logstash) and X-Pack 2012 2013 2014 2015 2016 24
Demo
Horizontal Scale Real-Time Data Availability Flexible Data Model Rapid Query Execution Sophisticated Query Language Schemaless 26
Tech Finance Telco Consumer 27
Security Alerting X-Pack Monitoring Reporting Graph Machine Learning 28
Elastic Cloud Available in AWS today Available in Google Cloud Platform (soon) Available as a private cloud/on-premise solution (Elastic Cloud Enterprise) 29
Elastic Cloud Enterprise 30
Elastic Stack X-Pack Elastic Cloud Application Search Metrics Analytics Log Analytics Business Analytics Security Analytics Many more 31
Search and analytics, it all started here More than 60% of our customers have a search or analytics use case 32
33
Logs Logs Logs, many devices, many systems More than 40% of our customers use our products for operational log analysis 34
Sniff sniff sniff, find the bad actors in your data 200% YoY growth in security use cases with our products 36
37 We mine and analyze 4 billion events every day to detect security hacks and threats.
75% of our customers use our products for multiple use cases LOG ANALYTICS METRICS SECURITY SEARCH OPERATIONAL ANALYTICS CUSTOM APPS 38
39 1,000+ developers use the Elastic Stack for use cases from trade tracking to creating new HR and compliance apps.
IT Operational Analytics Security Analytics Business Analytics Spiked 404 errors Unusual DNS activity Rare log messages Web attack Data exfiltration Failing sensor 41
Use Case Operational Analytics?? Error?
Use Case Security Analytics?? DNS?
Use Case Telemetry / Sensors ISP???
Where s the anomaly? Visual inspection is not practical 45
What s the right threshold? Rule-based alerts are insufficient 46
X-Pack 47
https://www.elastic.co/use-cases Discuss Web https://discuss.elastic.co Elastic{ON} https://www.elastic.co/elasticon/videos https://www.elastic.co/subscriptions 48
Thanks for listening! Q & A We re hiring! https://www.elastic.co/about/careers/ We re helping! https://www.elastic.co/subscriptions http://training.elastic.co