橡3_2石川.PDF

Size: px

Start display at page:

Download "橡3_2石川.PDF"

あいぞうもてぎ
4 years ago
Views:

1 PC RWC 01/10/31 2 1

2 SCore 1,024 PC SCore III PC 01/10/31 3 SCore SCore Aug Feb Oct Oct Oct April /10/31 4 2

SCore University of Bonn, Germany University of Heidelberg, Germany University of Tuebingen, Germany Oxford University, England Warwick University, England 01/10/31 5 Host PC RWC SCore III NEC

3 SCore University of Bonn, Germany University of Heidelberg, Germany University of Tuebingen, Germany Oxford University, England Warwick University, England 01/10/31 5 Host PC RWC SCore III NEC Express Servers Dual Pentium III 933 MHz 512 Mbytes of Main Memory # of Hosts 512 Hosts (1,024 Processors) Networks Myrinet Ethernet Links Linpack Result Gflops This is the world fastest PC cluster at August of /10/31 6 3

Myrinet-2000 2000 2 Gbps full duplex NIC Lanai DMA Engines HOST NIC Outgoing/Incoming Message 16 port switch warm hall routing 01/10/31 7 SCore Version 4 System Software High Performance

4 usec Round Trip time 228 MB/s Bandwidth PM/Ethernet Network Trunking Utilizing more than one NIC Global Operating System SCore-D Single/Multi User Environment Gang scheduling Checkpoint and restart

4 Myrinet Gbps full duplex NIC Lanai DMA Engines HOST NIC Outgoing/Incoming Message 16 port switch warm hall routing 01/10/31 7 SCore Version 4 System Software High Performance Communication Libs PMv usec Round Trip time 233 MB/s Bandwidth MPICH-SCore MPI Library 24.4 usec Round Trip time 228 MB/s Bandwidth PM/Ethernet Network Trunking Utilizing more than one NIC Global Operating System SCore-D Single/Multi User Environment Gang scheduling Checkpoint and restart Parallel Programming Language MPC++ Multi-Thread Template Library Shared Memory Programming Support Omni OpenMP on SCASH 10 times faster than Fast Ethernet + TCP/IP Three times as fast as Gigabit Ethernet + TCP/IP OMNI/SCASH SCASH PM/Shmem PM/Shmem driver Applications MPC++ MPICH-SCore SCore-D Global Operating System PM/Myrinet PM/Myrinet driver PMv2 Myrinet NIC PM firmware PM/Ethernet PM/Ethernet driver PVM-SCore Ethernet driver PM/UDP Socket UDP/IP Ethernet NIC PBS Linux User Level Kernel Level NIC Level 01/10/31 8 4

5 PM PM vs. GM DMA GM DMA 01/10/31 9 MPI Point to Point MPI Communication Bandwidth 2.50E+08 PM/Myrinet GM 1.0E+08 PM/Ethernet PM/Ethernet (2Way) 2.00E+08 LAM/MPI Bandwidth (Byte/sec) 1.50E E+08 Bandwidth (Byte/sec) 1.0E E E E+00 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 Message Size (Byte) 1.0E E E E E E E E E+07 Message Size (Byte) 01/10/

6 Application Benchmark 512x256x256 01/10/31 11 Application Benchmark IS (Class C) PM/Myrinet GM PM/Ethernet PM/Ethernet (2Way) TCP/IP(LAM) Total Mops Number of Procs 01/10/

7 Application Benchmark FT (Class C) FFT PM/Myrinet GM PM/Ethernet PM/Ethernet (2Way) TCP/IP(LAM) Total Mops Number of Procs 01/10/31 13 Application Benchmark Total Mops LU (Class C) PM/Myrinet GM PM/Ethernet PM/Ethernet (2Way) TCP/IP(LAM) SSOR(Symmetric Successive Over-Relaxation) CFD Number of Procs 01/10/

8 Application Benchmark MG (Class C) PM/Myrinet GM PM/Ethernet PM/Ethernet (2Way) TCP/IP(LAM) Total Mops Number of Procs 01/10/31 15 PC 01/10/

9 SCore III : 2 2 PC 2 Ethernet Myrinet Myrinet Clos128 Gigabit Ethernet 01/10/31 17 Myrinet E128 Switches are connected by eight port switches. #6 #4 #2 #0 #1 E128 E128 nodes nodes nodes nodes nodes nodes #3 #5 #7 01/10/

10 Ethernet eth1 Summit7i eth0 Summit7i #6 #4 #6 #2 #4 #0 #2 #0 #3 #1 #3 #7 #1 #5 #5 #7 01/10/ /10/

11 01/10/ /10/

12 01/10/ Ethernet 4 2 Ethernet PC 5 01/10/

13 (1/2) (800 Mbytes) RPM anaconda installation tool Score MBR The First Partition contains the installation image Other Partitions are empty SCore II NEC Express Servers (2U type) with Myrinet 5 SCSI disks minutes and half for one disk copy 01/10/31 25 (2/2) 1st stage 2nd stage IP Kickstart anaconda 01/10/

14 Each rack, Each modules, and All racks Score rcstest all-to-all Stressing Myrinet network in terms of network packets memory and Lanai Processor of Myrinet NIC Stressing processors and memory in Some initial hardware failures appear at at the the stress test!!!! 01/10/31 27 Connection between Myrinet Card and PCI bus slot Performance degradation We have not found the reason Connection between Myrinet Line Card and back-plane No communication CRC errors Connection between Myrinet Card and Cable No communication 01/10/

15 PC PC Ethernet Ethernet Linux Ethernet NIC Ethernet Switch 01/10/ /10/

16 PC PC 01/10/31 31 NIC 01/10/

17 PC vs vs 01/10/31 33 PC PC PC Linux SCore 01/10/

18 PC 19, 2, 7 SCore Omni OpenMP PC 01/10/31 35 Real World Computing Partnership is over, but SCore Development is continued 01/10/

develop

develop SCore SCore 02/03/20 2 1 HA (High Availability) HPC (High Performance Computing) 02/03/20 3 HA (High Availability) Mail/Web/News/File Server HPC (High Performance Computing) Job Dispatching( ) Parallel

橡3_2石川.PDF

橡3_2石川.PDF