Myrinet2000 ご紹介

Size: px

Start display at page:

Download "Myrinet2000 ご紹介"

なごみおおかわち
4 years ago
Views:

1 34 HPC -Myrinet- ES HPC

2 Out Line Myrinet

3 HPC

4 O.S. Computer Computer Computer Computer Computer Low-level Interconnection Network (transport & switching)

5 2-4 / / OS Linux OS? 5-20% Xeon ~4GB ~400MB/s GbE Myrinet I/O PCI CPU

6 (MSC Software; LSTC LS-Dyna, biomedical, ) (Fluent, Zeus-MP ) (MM5) (GAMESS) (Cactus) (CHARMM, AMBER, and others) (macromolecular modeling, gene sequencing)

7 Myrinet, ( ),, MPP ANSI (ANSI/VITA ) Q 5,000~ 10,000~

8 Myrinet Technology in the Large LSU SuperMike Cluster SuperMike cluster Atipa dual 1.GHz Xeon servers 10 km optical-fiber LINPACK performance 2207 Gigaflops TOP500 Supercomputer

9 Myrinet Technology in the Small CSPI Quad-PowerPC VME Signal-Processing Board CSPI LANai9 VME

10 TOP FSL LSU Buffalo VPlant Legend ANL CPlant HELICS TITECH Russia Titan Magi SCore Platinum NRL Prairiefire Kyoto Oklahoma Scripps Maui HPCC LosLobos New Zealand Amsterdam OSC Maine Korea FSL TITECH 15 of the top 100 Myrinet HyperFabric 140 of the use Myrinet technology

11 Myrinet? M3F-PCI64C: ~7µs PCIXD-2: ~6.3µs 2+2 Gb/s ( MB/s) Full-bisection Clos networks 192 CPU logp < 1µs - - CRC 2 Web Low-level API + TCP UDP/IP + MPI + VI + PVM + Sockets Myrinet/GbE M3-SW16-E

12 GbE GbE LANai-XM M3-E12 GbE 256

13 Myrinet -Link 2002/ 6 - LANai XM Myrinet port, long-range-myrinet port, GbE port, InfiniBand port Myrinet, long-range Myrinet, GbE, & InfiniBand. 2Q03 - PCI-X 1 /2 GM-2 2 x ( ) MB/s = 1GB/s PCI-X SerDes VLSI chips PCI-Express x Myrinet (multi-protocol) links

14 - (M3-SW16-2E2X4F) 2 GbE ports, 2 X ports over 50nm fiber, and 4 Myrinet ports.

15 Lanai XM To line card XBar16 port SerDes SerDes or or GbE PHY GbE PHY To line-card front-panel port To line card µc (JTAG) SAN network interface Send/recv DMA engines Control & memory initialize X network interface Send/recv DMA engines L-bus memory interface Modes -Myrinet - Program control long-range fiber -GbE - InfiniBand x72b x72b SRAM SRAM Lanai XM RISC 225MHz RISC & Memory This circuitry is repeated for each special line-card port.

16 16 Myrinet

17 12 Clos

18 Myrinet / Clos ( ) 12 M3-M SNMP CRC

19 Myrinet -Switch Myrinet, long-range-myrinet, GbE, ( ) InfiniBand GM XBar32. Clos switch 14U (?) 4x Myrinet links (2004~)

20 M3F-PCI64C-2

21 Myrinet Interface SAN port Network Interface Packet DMA Fast SRAM (2cycles/clock) Host DMA 64 bit RISC Bridge LANai chip Host PCI Bus

22 Myrinet Myrinet/PCI-X 225MHz x MB memory & 225MHz RISC; 1 port (Lanai XP) ~6.3µs GM latency. MPI, VI PCI-X 3Q/03: 333MHz x MB memory & 333MHz RISC; 2 ports (Lanai 2XP) ~4.µs GM latency. PCI-X PCI-X GM 2 Myrinet/PCI-Express 2004 Lanai 4 Myrinet ports (or 4x port) MHz~RISC & Memory

23 LANai XP- PCI-X SerDes SerDes PCI-card port X network interface Interface EEPROM & JTAG PCI-X bus Control & memory initialize PCI-X & DMA Engine Send/recv DMA engines L-bus memory interface RISC x72b x72b SRAM SRAM (225MHz) Lanai XP

24 LANai-XP M3F-PCIXD-2 PCI & PCI-X 3.3V only, Dual 2.4GHz Xeon / Serverworks chip set: PCI-X DMA read: 56 MB/s PCI-X DMA write: 1024MB/s

25 High End LANai 2XP PCI-X SerDes SerDes SerDes SerDes PCI-card port PCI-card port X network interface X network interface Send/recv DMA engines Send/recv DMA engines Interface EEPROM & JTAG Control & memory initialize L-bus memory interface x72b x72b SRAM SRAM PCI-X bus PCI-X & DMA Engine RISC (333MHz) Lanai 2XP

26 Myrinet Software Interface Applications MPI VIA Middleware Host OS UDP Ethernet IP TCP Myrinet OS APIs Multiple host processes) Myrinet Control Program(MCP) 10/100/1000 Mb/s Mb/s Myrinet

27 GM Fully protected CPU GM Data-Rate Performance (Myrinet-2000 Fiber Interfaces) UNIX user process to user process End-to-end data integrity GM short-message latency (Myrinet-2000 interfaces) ~ 7µs (PCI64C) or ~9µs (PCI64B) GM CPU overhead < 1µs per message (LogP)

28 GM OS Linux Win 2000/XP Solaris Tru64 AIX Irix * VxWorks * MacOS X FreeBSD, Platforms IA-32, IA-64, Alpha, PowerPC, IBM Power 3 & 4 IA32, IA-64 UltraSPARC Alpha IBM Power MIPS PowerPC Apple Macintosh G4 IA-32 & Alpha

29 GM GM API Low TCP/IP Ethernet GM 1.9 Gb/s TCP/IP (netperf ) MPICH-GM Argonne MPICH over GM VI-GM VI Architecture API directly over GM Sockets-GM UNIX Windows sockets (or DCOM) over GM. PVM SCore

30 Myrinet - GM 2, GM GM-1 gm_get Ethernet CPU Myrinet-GbE Myrinet Express (MX): MPI Ethernet PCIXE 3.5µs (?) MPI 3Q03 Linux, PCIXD & PCIXE GM

橡3_2石川.PDF

橡3_2石川.PDF PC RWC 01/10/31 2 1 SCore 1,024 PC SCore III PC 01/10/31 3 SCore SCore Aug. 1995 Feb. 1996 Oct. 1996 1997-1998 Oct. 1999 Oct. 2000 April. 2001 01/10/31 4 2 SCore University of Bonn, Germany University