OpenStack Sahara Road to Kilo www.miran(s.com/jp Copyright 2015 Mirantis, Inc. All rights reserved
Agenda Hadoop Sahara Kilo Q&A Copyright 2015 Mirantis, Inc. All rights reserved Page 2
Hadoop Open-source ecosystem for reliable, scalable, distributed computing 安定性 拡張性的の分散処理のオプンソース エコシステム Copyright 2015 Mirantis, Inc. All rights reserved Page 3
: Hadoop : Yahoo! Apache Google : Apache Copyright 2015 Mirantis, Inc. All rights reserved Page 4
: Hadoop http://hortonworks.com/hadoop/ Copyright 2015 Mirantis, Inc. All rights reserved Page 5
: HDFS Data Locality.. ETL (Extraction, Transformation, Load) Copyright 2015 Mirantis, Inc. All rights reserved Page 6
: Web HVAC( ) Copyright 2015 Mirantis, Inc. All rights reserved Page 7
: / vs. / IDC EMC 40 zettabyte 1 70-80% Source A differs from Source B Dirty Data 1 http://ikt.hia.no/perep/eip_ind.pdf Copyright 2015 Mirantis, Inc. All rights reserved Page 8
: Databases Data Source DB Hadoop Cluster Result Data Source Hadoop Cluster Result DB https://hadoop.apache.org/ Copyright 2015 Mirantis, Inc. All rights reserved Page 9
: (Computation) (Network, Storage) Ask the right question Copyright 2015 Mirantis, Inc. All rights reserved Page 10
Sahara OpenStack Big Data as a Service Copyright 2015 Mirantis, Inc. All rights reserved Page 11
Sahara: Why To make cluster management and data analysis tools simpler, agile and accessible to users. Copyright 2015 Mirantis, Inc. All rights reserved Page 12
Sahara: Savanna (0.3 ) Hortonworks RedHat 2013 (Portland) (PTL Sergey Lukjanov) Juno (2014) OpenStack https://launchpad.net/sahara Copyright 2015 Mirantis, Inc. All rights reserved Page 13
Sahara: Sahara Glance VM Distribution and version drivers Node Groups: Master nodes, Computation nodes.. Cluster: node Groups Copyright 2015 Mirantis, Inc. All rights reserved Page 14
Sahara: QA EDP Copyright 2015 Mirantis, Inc. All rights reserved Page 15
Sahara: Hadoop Hadoop REST API ( ) Swift OpenStack (Horizon) Hadoop (Vanilla Apache, Hortonworks, Cloudera, Spark) Copyright 2015 Mirantis, Inc. All rights reserved Page 16
Sahara: Use Case : EDP EDP (Elastic Data Processing) Sahara Job Data Source Job Binary Job ( )Job Job Execution Job Copyright 2015 Mirantis, Inc. All rights reserved Page 17
Sahara: EDP Job Execution (AWS EMR ) Oozie, Spark Pluggable data sources Swift API, HDFS, *Trove, *Ceph job Jar, Pig, Hive UI Hive Pig Job Copyright 2015 Mirantis, Inc. All rights reserved Page 18
Sahara: Horizon Sahara Dashboard Sahara Python Client REST API Keystone Auth Cluster Configuration Manager Job Sources Job Manager Data Access Layer Vendors Plugins Data Sources Resources Orchestration Manager Hadoop VM Hadoop VM Swift Trove DB Heat Hadoop VM Hadoop VM Nova Glance Cinder Neutron https://wiki.openstack.org/wiki/sahara Copyright 2015 Mirantis, Inc. All rights reserved Page 19
Mirantis OpenStack Sahara VM Hadoop cluster Direct disk mapping increases IO performance (1-2% difference against BM) Kilo/MOS 7.0 Fuel ( ) HEAT Sahara Copyright 2015 Mirantis, Inc. All rights reserved Page 20
Sahara: (KILO) MapR plugin Storm plugin Native Ceph Ironic ( ) Sahara OpenStack Sahara http://specs.openstack.org/openstack/sahara-specs/ Copyright 2015 Mirantis, Inc. All rights reserved Page 21
: 2, 400 5,000 : CDH ( ) : MOS 5.1 CDH plugin QA Copyright 2015 Mirantis, Inc. All rights reserved Page 22
デモンストレーション MOS 6.0 Juno with Sahara Copyright 2015 Mirantis, Inc. All rights reserved Page 23
Q&A How can we help you succeed with OpenStack? Copyright 2015 Mirantis, Inc. All rights reserved Page 24
We are looking for you 一 緒 にOpenStackを 広 めませんか? チームメンバーを 募 集 中! info.jp@miran,s.com トレーニングサービスもご 利 用 ください! Copyright 2015 Mirantis, Inc. All rights reserved Page 25