1 Docker maebashi@iij.ad.jp
Docker 2015 Internet Initiative Japan Inc.
http://www.iij.ad.jp/biz/storage/ 2015 Internet Initiative Japan Inc.
IIJ GIO & REST API(AWS S3 ) + Hadoop/Hive 2015 Internet Initiative Japan Inc.
IIJ GIO data storage API data data data query analysis API container 2015 Internet Initiative Japan Inc.
Hadoop + Hive 2015 Internet Initiative Japan Inc.
( ) DN analysis API query NN RM Hive metastore NM task DN NM task storage API NN: Name Node RM: Resouce Manager DN: Data Node NM: Node Manager 2015 Internet Initiative Japan Inc.
2015 Internet Initiative Japan Inc.
Hypervisor Hive (UDF )
Docker?? Linux Namespaces, Cgroups, Capabilities
vs App A App B Bins/Libs Bins/Libs Guest OS Guest OS Hypervisor Host OS Server Hypervisor App A App B Bins/Libs Bins/Libs Docker Engine Host OS Server Docker OS
Docker MapReduce HiveQL Docker
Docker
Docker
Docker ( )
Container Container Container Docker Container Container Docker Container Container Container Docker Container Container Container
Docker Swarm Kubernetes Apache Mesos Fleet Nomad...
doma(docker manager)? Docker analysis API
doma analysis API HTTP LB(nginx) master API IP etc request / response master DB (MySQL MHA) HTTP HTTP over Unix domain socket slave slave API Docker Remote API Docker daemon query Container docker
master ( ) Ruby + Rack + EventMachine
1 1 1 2 3 Docker Container Container Docker Container Container Container Docker Container Container Container
master docker CPU IP 1 ( )IP
(CPU ) docker
slave Docker daemon wrapper master Docker daemon API slave API Go master HTTP over Unix domain socket slave Docker Remote API Docker daemon
Docker Remote API HTTP REST JSON docker API Docker client docker pull docker run docker... Docker Remote API HTTP over Unix domain socket TCP Host Docker daemon container container container
slave iptables
cgroup CPU cpuset memory cgroup
CPU (Docker Remote API )cgroup cpuset.cpus CPU : "CpusetCpus":"0-2,7" CPU
loopback mount mkfs sparse tar mount(0.1 0.2 ) 1
Docker container network namespace eth0 eth0 (veth) host network namespace vetha vethb docker0 NIC
Docker eth0 eth0 eth0 eth0 vetha vethb vetha vethb docker0 docker0 NIC NIC ( EXPOSE Hadoop )
pipework weave flannel libnetwork overlay driver...
docker ("NetworkDisabled": true) slave
docker eth0 eth0 container network namespace eth0 eth0 vetha vethb vetha vethb docker0 host network IP namespace NIC NIC bridge0 host-veth IP host-geth
eth0 eth0 host-geth eth0 eth0 host-geth IP master slave
iptables CAP_NET_RAW
Docker Docker HUB repository :tag (docker 1.6 は Content Addressable Image Identifiers ) OS febootstrap CentOS 6 OS JDK, Hive, Hadoop Dockerfile
( ) fluentd ( ) ( Docker log driver )
doma analysis API
slave cgroup /proc/net/dev ( cadvisor )
コンテナのモニタリング CPU Accounting Network traffic Memory 46
OS CentOS 6 Docker 1.8 CentOS 6 Red Hat Docker RHEL 6 OS CoreOS, Project Atomic, Snappy, RancherOS
Hadoop/Hive Docker Docker
doma(docker manager) Kubernetes, Swarm, Mesos libnetwork, flannel, weave fluentd docker log driver cadvisor CentOS 6 CoreOS, Atomic Host