スケールアウト製品の展望

HP Tech Power Club HP Technology At Work 2015 スケールアウト分科会 スケールアウト製品の展望 日本ヒューレット パッカード株式会社プリセールス統括本部サーバー技術本部 HPC 担当部長 畑崎隆雄

畑崎隆雄 経歴 コンピューターハードウェアの設計からコンパイラ 並列計算用通信ライブラリ開発などを経て 1999 年より現職 最近では東工大 TSUBAME スーパーコンピューターの設計から導入までを担当 専門分野 大規模 HPC システム設計 プログラム高速化など 畑崎隆雄 日本ヒューレット パッカード株式会社プリセールス統括本部サーバー技術本部 HPC 担当部長 私のこだわり スケールアウトシステムはさまざまな技術の複合体です そして HP はさまざまな先端技術を持つグローバル企業です 多くの技術チームの連携が HP の強みです 2

Hyperscale Market への取り組み

Hyperscale 市場は他市場の 10 倍の速度で成長中であり 2017 年には全世界サーバー市場の半分を占める Service provider だけでも市場の 1/3 以上に $50 Traditional Private Cloud HPC Service Provider $40 $30 36% 50+% $20 $10 $0 2013 2014 2015 2016 2017 4

HP Hyperscale Strategy Service Provider Optimized The Big Data Revolution Bridging Enterprise and Hybrid Cloud Supercomputing for the Mainstream Radically Cost-Optimized, Open Designs for Extreme Scale HP Cloudline Software-defined ProLiant Moonshot Storage Density ProLiant SL4500 Scalable Multi-node ProLiant SL2500 Rack Scale Efficiency ProLiant Apollo 6000 and SL6500 Delivering breakthrough efficiency at scale Performance Density and Green Cooling ProLiant Apollo 8000 5

低消費電力サーバー Moonshot

Redefining what s possible HP Moonshot System Optimize application performance A system that truly fits the workload it is running Realize breakthrough economics Significant power, space and cost savings Accelerate business innovation Freedom to create new business models 8

Moonshot Leap 4 Launched Q3 2014 What s New? The world s first 64 bit ARM server & developer program Redefining the economics of web infrastructure ARM64 Web Caching ProLiant m400 Moonshot ARM64 SW Developer Program Web- Infrastructurein-a-box ProLiant m300 Web Hosting ProLiant m350 Scott 4 Accelerating customer innovation around new workloads Enhancing Moonshot enterprise class capabilities Video Transcoding ProLiant m710 Application Delivery ProLiant m710 Real-time Data Processing ProLiant m800 10Gb networking Moonshot-45XGc Switch Module Standardized manageability through RESTful API, Graphical UI External Storage Support via iscsi protocol 9

10

Big Data 向けプラットフォーム SL4500

HP ProLiant SL4500 Server Series Flexibility to address Big Data workloads in any data center SL4540 (1X60) Object Storage SL4540 (2X25) MS Exchange SL4540 (3X15) Hadoop 1 node configuration 1 compute node with up to 60 LFF and 2 SFF hard drives 2 node configuration 2 compute nodes with up to 25 LFF and 2 SFF hard drives per node 3 node configuration 3 compute nodes with up to 15 LFF and 2 SFF hard drives per node Note: HP ProLiant SL4540 Gen8 Server and HP ProLiant SL4545 G7 Server will be available in 1, 2 and 3 node based configurations as shown above 12

HP ProLiant SL4500 Series Key features and benefits Key feature Up to 540 LFF hard drives (SAS/SATA/SSD) with up to 27compute nodes per 42U rack Includes HP Smart Array technology Cluster management, Power management and 94% efficiency Platinum Plus power supplies Shared power, cooling and management Benefit Range of core to spindle ratios optimized for every big data workload Scalable performance across multiple drive technologies (SSD, SAS or SATA) Enterprise class solution for management your big data workloads Energy savings, reduced complexity with fewer cables 13 1Calculated Copyright 2015 using Hewlett-Packard 4TB HDD, Development which will Company, be available L.P. The information early FY2013, contained herein 9 is x subject SL4540/SL4545 to change without servers notice. in a 42U rack

HP ProLiant SL4500 Series Flexible IO Bay Flexible PCI slot expansion options 14 Low profile Supports network card, HP I/O accelerator, or 2nd Smart Array controller (future) Connectivity to outside world Dedicated IO Bay per node Contains Smart Array p420i Gen8 controller 2x1Gb Ethernet Optional 2x10Gb Ethernet options High speed signaling Up to 24 lanes PCI Express Gen2 Up to16 lanes (future) of SAS/SATA (8 current) PCIe Slot 2x RJ45 1GbE Optional ilo QSFP 10GbE Or 40G QDR IB SFP+ 10GbE Serial Port

16

Rethinking Analytics Infrastructure Contrary to initial expectations, the file system servers and the accompanying Gb Ethernet plumbing were not only able to transfer data efficiently between the processing nodes and the storage tier, but were able to demonstrate efficient (ability to efficiently use all resources with no signs of bottlenecks or stranded resources) processing of mixed analytics workloads across the cluster. 17

高性能 高密度実装サーバー Density-Optimized Apollo 6000

47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 Speed: Green=1000Mbps, Yellow=10/100Mbps Duplex: Green=Full Duplex, Yellow=Half Duplex 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Speed: Green=1000Mbps, Yellow=10/100Mbps Duplex: Green=Full Duplex, Yellow=Half Duplex 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Speed: Green=1000Mbps, Yellow=10/100Mbps Duplex: Green=Full Duplex, Yellow=Half Duplex 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Speed: Green=1000Mbps, Yellow=10/100Mbps Duplex: Green=Full Duplex, Yellow=Half Duplex 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 10/100/1000Base-T 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 10/100/1000Base-T 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 10/100/1000Base-T 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 10/100/1000Base-T 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 49 50 49 50 49 50 53 Unit HP A5800 Series Switch JG225A Green = Simplex Yellow = Duplex 51 52 SFP+ Green=10Gbps, Yellow=1Gbps 51 52 53 51 52 SFP+ Green=10Gbps, Yellow=1Gbps 53 51 52 SFP+ Green=10Gbps, Yellow=1Gbps 53 Unit HP A5800 Series Switch JG225A Green = Simplex Yellow = Duplex 54 54 Unit HP A5800 Series Switch JG225A Green = Simplex Yellow = Duplex 54 HP A5800 Series Switch JG225A Unit 54 Green = Simplex Ye low = Duplex SFP+ Green=10Gbps, Yellow=1Gbps Mode SYS Mode SYS Mode SYS Mode SYS 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 XL220a Special design for Intel EDA 2P CPU CPU CPU 2x 1P CPU DIMMs DIMMs DIMMs ラックあたり 8* 新設計エンクロージャ 160 サーバー 2* パワーシェルフ 4x HP 5900AF-48GX-4QSFP+ Switch 2009 年に 38,000 台の BL460c を導入 ( ラックあたり 64 サーバー ) 2013~2014 年に約 30,000 サーバー導入 19

HP Apollo 6000 標準の奥行き 1 メートルラック 後部配線 Features 1 slot and 2 slot tray support -10 single slot trays -5 double slot trays Mix-n-match trays Shared cooling 12V DC power distribution 5U tall infrastructure Up to 5700W per chassis 5U 22.0 cm Serviceability Front serviceable trays Standard rear cabling Front serviceable hot plug drives Redundant, hot plug fans 5U (H) x 44.81cm (W) x 86.23cm (D) 5U (H) x 17.64 in (W) x 33.95 in (D) 21 SL6500 96.9 cm (D) DL380 25 SFF - 69.85 cm (D) DL380 12 LFF 74.94 (D)

HP Apollo 6000 Infrastructure Rack Level Power Efficiency Network I/O Modules Two 1GbE 1-port module OR Two FlexibleLOM riser: supports 1GbE, 10GbE or IB Management Module 22 Power Shelf HP APM Aggregated ilo

HP Apollo 6000 パワーシェルフ c7000 用高効率 (Platinum) 電源を利用 Front View 1.5U 6.5 cm 44.8 cm Efficiency External pooled power shelf Fits up to 6 power supplies 2400W or 2650W power supplies Up to 15.9kW non-redundant Single or 3-phased AC input Up to twelve 12V DC cables 78.4 cm 1.5U (H) x 44.81cm (W) x 78.44cm (D) 1.5U (H) x 17.64 in (W) x 30.88 in (D) Back View 23

HP ProLiant XL250a Server 2P Compute with Accelerators What s new: 2p Server with up to 2 accelerators NVIDIA Tesla K40 or new K80 GPUs Intel Xeon Phi 5110p or 7120p Coprocessors Up to 6 SFF hot plug SAS/SATA/SSD Up to 5 servers per 5u chassis, 35 servers per 42u rack 24

Accelerating Performance New Intel and NVIDIA accelerators designed for HPC 4x teraflops per square foot Customer Benefits Fully integrated accelerators, installed, tested and supported in HP Apollo solutions Key features Support for Intel Xeon Phi 5110P and 7120p Coprocessors Accelerate application performance, choosing the right accelerator for each application Enable maximum performance/ft 2 and performance/watt in HP Apollo solutions Support for NVIDIA Tesla K40 and New K80 GPUs Unique to HP: Support for NEW NVIDIA Tesla K40d GPUs and Intel Xeon Phi 7120D Coprocessors 25

https://supcom.hgc.jp/japanese/sys_const/system-main_s3.html 26

HP Advanced Power Manager A smart and optimal solution to manage racks in your datacenter. Compatible with Apollo 6000 System SL6500 SL4500 SL2500 Moonshot 1500 System Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Converged Manageability for Hyperscale In the same manner that HP built integrated manageability into our c-class enclosures We can offer the same converged experience at the rack level. 28

HP APM - Access Ports Serial Concentrator Consolidated Management Cable Ports Power Device Management (PDM) Ports Micro SD Slot Ethernet and Serial Management Access Consolidated ilo Port SL-APM Ports (for backward compatibility) EPO, OnGen, and Buddy Reset ports 29

Component Connectors Apollo 8000 Apollo a6000 chassis Moonshot s6500 Apollo 6000 power shelf Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Autobot (Li-Ion UPS) SL4540 Endora G2 (UPS)

APM Automatic Power Capping データセンター内の電力割り当ての最適化を支援するための仕組み 管理者が定義する Zone に対して電力割り当てを設定する APM は継続的に消費電力を監視し 必要に応じて電力割り当て上限を動的に変更する 割当量は c-class ではエンクロージャ単位で実装されている Dynamic Power Capping と同様のアルゴリズムで決定される サーバーの消費電力限界に対して 現在の負荷がどの程度か どれくらい Capping に貢献しているか 貢献度に応じて 余裕 を割り当てる 31

超高密度ハイエンド水冷システム Apollo 8000

33

NREL におけるエネルギー再利用 Warm-water cooling 蒸発冷却塔 常温水冷却コンピュータシステム 融雪設備 エネルギー再利用循環系 施設暖房 34

冷却効率を具体的に比較してみる 水が圧倒的に有利 0.58 bhp Fan 0.05 bhp Pump 4 gpm 35

HP Apollo 8000 Coolant Distribution Unit (CDU) およびラック間配管キット付き Basic Building Block 2 Compute racks + 1 CDU with the plumbing kit dry disconnect による常温水冷却 Rack as a chassis サーバートレイ内からは水を遮断する構造 Compute rack Water wall が冷たくなる CDU Underfloor plumbing kit この側面が熱くなる 36

漏水対策 (2) Valve (0.5PSI ) Filter (1-4 PSI ) 20PSI HEX (4-12PSI ) 5PSI icdu Vacuum pump Sub-atmosphere Water wall (1-4PSI ) Valve (0.5PSI ) 37

最新設計の三相大電圧入力 大規模構成では変換損失も無視できない Typical UPS Server 480VAC Source 480VAC HVDC HVDC 480VAC 208VAC 208VAC HVDC 12VDC 480-208 PSU PSU Rectifier Batteries Inverter PDU XFMR Rectifier Step-down IT Load Apollo 98% conversion * 99% losses * 98% conversion * 97% conversion * * 97% * 98% 99% distribution conversion conversion = 87% efficiency 480VAC Source 480V Titan Rectifier Rack HVDC BBU Hyperion DC-DC Server 12VDC IT Load 38 98% 97% conversion Alternate conversion inputs * = 95% efficiency

まとめ Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

まとめ スケールアウト市場は HP が会社として注力している分野 Low-end から High-end まで さまざまなハードウェア製品の開発に積極的に投資している IoT や Big Data といった New Style of IT のためのハードウェアとしては Moonshot や SL4500 がある 新しいシステムアーキテクチャの一例として Hadoop への応用がある Apollo 6000 は 大型案件のための特別設計から生まれた高性能 高密度実装製品 パワーシェルフの外だしにより 大規模システムにおけるコスト削減を加速 ブレードサーバーのシャシの概念をラック単位に拡張 消費電力制御のためには Advanced Power Manager が有用 次世代の High-end システムには水冷が必須となる さまざまな新技術を採用して業界の技術賞を受賞した Apollo 8000 は エネルギー再利用までを考慮した High-end 水冷システム 41

Thank you Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.