THE PARALLEL Issue UNIVERSE James Reinders Parallel Building Blocks: David Sekowski Parallel Studio XE Cluster Studio Sanjay Goil John McHug

Size: px

Start display at page:

Download "THE PARALLEL Issue UNIVERSE James Reinders Parallel Building Blocks: David Sekowski Parallel Studio XE Cluster Studio Sanjay Goil John McHug"

うのすけさわなか
5 years ago
Views:

1 THE PARALLEL Issue UNIVERSE James Reinders Parallel Building Blocks: David Sekowski Parallel Studio XE Cluster Studio Sanjay Goil John McHugh

2 JAMES REINDERS 3 Parallel Studio XE Cluster Studio SANJAY GOIL JOHN MCHUGH 5 Parallel Studio XE Cluster Studio Windows* Linux* C/C++ Fortran Parallel Building Blocks: DAVID SEKOWSKI 17 3 TBB Cilk Plus Array Building Blocks ArBB Array Building Blocks MICHAEL MCCOOL 23 Array Building Blocks ArBB SIMD MKL GREG HENRY SHANE STORY 29 MKL hotspot Print : DON GUNNING NICK MENG PAUL BESL 31 / print 2010 Intel Corporation. Intel Intel Intel Core Itanium Xeon Intel Corporation * 2

3 James Reinders Intel Threading Building Blocks: Outfitting C++ for Multicore Processor Parallelism Parallel Universe 1 SIMD 2010 / 3

2010 11 9 Parallel Studio XE Cluster Studio 2 Parallel Studio XE C C++ Fortran Cluster Studio MPI C C++ Fortran 12.0 VTune MPI C/C++ Parallel Building Blocks Co-Array Fortran.

4 Parallel Studio XE Cluster Studio 2 Parallel Studio XE C C++ Fortran Cluster Studio MPI C C++ Fortran 12.0 VTune MPI C/C++ Parallel Building Blocks Co-Array Fortran.NET Parallel Studio XE Linux* Windows* Composer XE VTune Amplifier XE Inspector XE Cluster Studio Linux* Windows* Composer XE MPI MPI / OpenMP* MPI Co-Array Fortran MKL IPP TBB Cilk Plus Array Building Blocks ArBB Parallel Building Blocks PBB Parallel Building Blocks Fortran MKL / Intel Inspector XE: An essential tool during development along with Intel Composer XE Inspector XE: Composer XE Inspector XE hotspot hotspot VTune Amplifier XE MPI MPI On a path to petascale with commodity clusters and Intel MPI MPI HPC MPI / Parallel Studio XE Cluster Studio JAMES REINDERS

5 Parallel Studio XE Cluster Studio Sanjay Goil John McHugh 5

Parallel Studio XE 2011 Composer XE Inspector XE VTune Amplifier XE 1 2010 9 Parallel Studio 2011

Fortran Parallel Studio XE 2011 C/C++ Fortran MKL IPP PBB TBB Cilk Plus ArBB Inspector XE / VTune

6 Parallel Studio XE 2011 Composer XE Inspector XE VTune Amplifier XE Parallel Studio 2011 Microsoft* Visual Studio* Windows* C++ Parallel Building Blocks PBB C/C++ Windows*/Linux* C/C++ Fortran Parallel Studio XE 2011 C/C++ Fortran MKL IPP PBB TBB Cilk Plus ArBB Inspector XE / VTune Amplifier XE HPC 10 HPC Cluster Studio MPI HPC MPI / C/C++ Fortran 1 / 1 IA MPI HPC HPC 2 / 6

Composer XE C/C++ Fortran > / Inspector XE / > VTune Amplifier XE > 2 Parallel Studio XE 2011 > : Parallel Studio XE Windows* Linux* Mac OS* X C/C++ Fortran > : Inspector XE / > : VTune Amplifier XE

7 Composer XE C/C++ Fortran > / Inspector XE / > VTune Amplifier XE > 2 Parallel Studio XE 2011 > : Parallel Studio XE Windows* Linux* Mac OS* X C/C++ Fortran > : Inspector XE / > : VTune Amplifier XE MKL IPP / > : Parallel Studio XE > : Composer XE AVX C/ C++ PBB Fortran Fortran 2008 Co-Array Fortran > : Parallel Studio XE 1 hotspot / 7

8 Composer XE C++ C++ XE Visual Fortran Visual Fortran Composer XE Visual Fortran IMSL* Visual Fortran Composer XE IMSL* VTune VTune Amplifier XE Inspector XE Cluster Studio 3 Parallel Studio XE 2011 Windows*/Linux* x86 C/C++ Fortran 3 8

Jorge Martinis 4 SIMD 5 9 http://software.

9 Parallel Studio XE 2011 Windows* C++ Cilk Plus Parallel Studio XE 2011 BR&E Inc. Jorge Martinis 4 SIMD 5 9

hotspot BlueJeans Network IPP 7.0 / IPP IPP 7.0 CPU BlueJeans Network Emmanuel Weber 6 Composer XE Composer XE C/C++ Fortran (v12.0) MKL 10.3 IPP 7.0 TBB 3.0 2 C/C++ C++ XE 12.

10 hotspot BlueJeans Network IPP 7.0 / IPP IPP 7.0 CPU BlueJeans Network Emmanuel Weber 6 Composer XE Composer XE C/C++ Fortran (v12.0) MKL 10.3 IPP 7.0 TBB C/C++ C++ XE 12.0 AVX Sandy Bridge PBB Cilk Plus TBB ArBB 4 AVX SIMD x86 (GAP) Windows* Visual Studio* 2010 Fortran XE 12.0 x86 Fortran 2003 Fortran 2008 (Co-Array Fortran AVX ) 5 / MKL 10.3 AVX LAPACK C IPP 7.0 AVX AES 10

Windows* Windows* Linux* 6 Inspector XE / 7 / / C/C++ Fortran Inspector

11 Inspector XE > Inspector XE 2011 OTRADA Inc. CEO CTO Alex Migdalski 7 / Parallel Studio XE 2011 Parallel Studio Linux* Windows* Windows* Linux* 6 Inspector XE / 7 / / C/C++ Fortran Inspector XE / : > > > > > > Windows*/Linux* GUI > 11

12 8 VTune Amplifier XE - hotspot > > CPU VTune Amplifier XE - VTune Amplifier XE 2011 VTune... > > {+} 9 VTune Amplifier XE 2011 VTune Windows*/Linux* 8 9 VTune : > > hotspot > > > > Windows* > > > > Visual Studio > root (Linux*) > EBS root 12

Inspector XE - > > (SSA) SSA Envivio Mikael Le Guerroue Parallel Studio XE 10 Parallel Studio XE 2011 10 11 Parallel

13 Inspector XE - > > (SSA) SSA Envivio Mikael Le Guerroue Parallel Studio XE 10 Parallel Studio XE Parallel Studio XE (SSA) > > > > 250 > > > > Windows*/Linux* GUI 13

Linux* Windows* Windows* Linux* C/C++ Parallel Building Blocks C/C++ Co-Array Fortran (CAF) Fortran 2008 Fortran Fortran Cluster Studio 2011 Fortran 1 GUI 11 Cluster Studio > Composer XE > MPI > /

14 Linux* Windows* Windows* Linux* C/C++ Parallel Building Blocks C/C++ Co-Array Fortran (CAF) Fortran 2008 Fortran Fortran Cluster Studio 2011 Fortran 1 GUI 11 Cluster Studio > Composer XE > MPI > / HPC Cluster Studio 2011 IA MPI MPI 4.0 / 8.0 MPI C/C++ Fortran MKL 10.3 IPP 7.0 PBB Composer XE Cluster Studio 2011 Co-Array Fortran (CAF) (ICR) IA (TCO) Cluster Studio 2011 MPI MPI OpenMP* PBB Cluster Studio 2011 C/C++ Fortran Linux* Windows* 1 14

15 PO 12 / > > > > hotspot Cluster Studio 2011 > : MPI > : Composer XE C/C++ PBB Fortran Fortran 2008 Co-Array Fortran SIMD > MPI : / MPI > : Windows* Linux* > : IA HPC (TCO) > : Cluster Studio

MPI MPI InfiniBand* (IB) MPI MPI 1 InfiniBand (IB) C/C++ Parallel Building Blocks SMP C/ C++ Coarray Fortran (CAF) Fortran 2008 Fortran Linux* Windows* Fortran Fortran 2008 Fortran 2003 MKL IPP

16 MPI MPI InfiniBand* (IB) MPI MPI 1 InfiniBand (IB) C/C++ Parallel Building Blocks SMP C/ C++ Coarray Fortran (CAF) Fortran 2008 Fortran Linux* Windows* Fortran Fortran 2008 Fortran 2003 MKL IPP Linux* Windows* Windows* Linux* 13 Parallel Studio XE Cluster Studio Windows* Linux* C/C++ Fortran Parallel Studio XE 2011 C/C++ Fortran MKL IPP PBB TBB Cilk Plus ArBB Inspector XE / VTune Amplifier XE Cluster Studio 2011 MPI / C/C++ Fortran MKL IPP PBB TBB Cilk Plus ArBB : 16

17 Parallel Building Blocks David Sekowski 17

18 : 5 C++ TBB Adobe* TBB TBB Parallel Building Blocks PBB > > HPC Andy Grove Only the Paranoid Survive 1 / CPU / > > CPU > > > 1 1 > 1 > 18

19 1 / MPI 3 2 AI I/O for for ( ) : SIMD (Single Instruction Multiple Data) & SIMD

20 2 : (1) (2) 3 OS 3 ( ) 20

21 Parallel Building Blocks - OS - Parallel Amplifier Parallel Inspector OS HW IDE 4 OpenMP* OpenCL OpenMP* Fortran C OpenMP* OpenMP* Web OpenMP* Parallel Building Blocks TBB TBB C++ TBB API TBB (1) (2) Cilk Plus Array Building Blocks ArBB 5 Cilk Plus C/C++ C/C

22 TBB Cilk Plus Windows* Microsoft* C++ Linux* GCC Cilk ++ SDK ArBB intel-array-building-blocks 5 ArBB Ct TBB C++ JIT CPU C++ ArBB ArBB 3 TBB Cilk Plus ArBB Parallel Building Blocks PBB Parallel Studio 2011 Parallel Studio XE 2011 PBB ArBB Ct TBB C++ 22

23 Array Michael McCool Building Blocks

Array Building Blocks ArBB Parallel Building Blocks ArBB Microsoft* GNU C++ ArBB http://intel.com/go/arbb Windows* Linux* ArBB?

24 Array Building Blocks ArBB Parallel Building Blocks ArBB Microsoft* GNU C++ ArBB Windows* Linux* ArBB?? ArBB SIMD ArBB API SIMD 2 SIMD SIMD SIMD SIMD Parallel Building Blocks PBB SIMD SIMD SSE 1 SSE 4 AVX SIMD 8 (MIC) SIMD 16 24

Parallel Building Blocks PBB 3 SIMD SIMD 2 1 SIMD 2 SIMD SSE AVX MIC AVX SSE SSE AVX 2 1 C++ 1 SSE AVX AVX SIMD SIMD SIMD C/C++ Parallel Building Blocks PBB 3 1 MKL IPP 2 1 C/C++ Cilk Plus C/C++ 2

25 Parallel Building Blocks PBB 3 SIMD SIMD 2 1 SIMD 2 SIMD SSE AVX MIC AVX SSE SSE AVX 2 1 C++ 1 SSE AVX AVX SIMD SIMD SIMD C/C++ Parallel Building Blocks PBB 3 1 MKL IPP 2 1 C/C++ Cilk Plus C/C++ 2 ArBB ArBB C++ API ISO C++ C++ - C/C++ ArBB ArBB ArBB C++ API ArBB C++ float) integer) f32 i32 32 C++ ArBB dense<t,d> T D D 1 T ArBB ArBB 25

.. call(doit)(a,b,c,d); call 1 ArBB call C++ ArBB C++ C++ ArBB ArBB ArBB call capture call C++ ArBB call BLOG highlights

26 ArBB 2 A B C D 4 dense<f32> A += (B/C) * D; ArBB call void doit(dense<f32>& A, dense<f32> B, dense<f32> C, dense<f32> D) { A += (B/C) * D; }... call(doit)(a,b,c,d); call 1 ArBB call C++ ArBB C++ C++ ArBB ArBB ArBB call capture call C++ ArBB call BLOG highlights Cilk Plus ABI JAMES REINDERS Cilk Plus Cilk Plus ABI cilk.com Cilk Plus Windows* Linux* Cilk Plus Cilk Plus 2009 Cilk Arts JAMES Go-Parallel.com Go Parallel : Translating Multicore Power into Application Performance 26

$call map 2 ArBB prefix scan) / / ) 1 1 2 A B map call map f32 A _ dot _ B = add _ reduce A * B); void doit(dense<f32>& A, dense<f32> B, dense<f32> C, dense<f32> D) { map(kernel)(a, B, C, D); }$

27 void doit dense<f32>& A, f32 b, f32 c, dense<f32> D) { map(kernel)(a, b, c, D); } call(doit)(a,b,c,d); ArBB void kernel(f32& a, f32 b, f32 c, f32 d) { a += (b/c)*d; } kernel b c f32 kernel A D b c call map 2 ArBB prefix scan) / / ) A B map call map f32 A _ dot _ B = add _ reduce A * B); void doit(dense<f32>& A, dense<f32> B, dense<f32> C, dense<f32> D) { map(kernel)(a, B, C, D); } call(doit)(a,b,c,d); C++ 1 ArBB ArBB 27

$capture ArBB C++ ArBB (_end_for ) _for (;) (,) ArBB / ArBB STL bind ArBB C++ call doit map 1 2 1 1 1 ArBB int max _ count = MAX _ COUNT; void mandel(i32& d, std::complex<f32> c) { i32 i; std::co m$

28 capture ArBB C++ ArBB (_end_for ) _for (;) (,) ArBB / ArBB STL bind ArBB C++ call doit map ArBB int max _ count = MAX _ COUNT; void mandel(i32& d, std::complex<f32> c) { i32 i; std::co m plex<f32> z = 0.0f; _ for (i = 0, i < max _ count, i++) { _ if (abs z) >= 2.0f) { _ break; } _ end _ if; z = z*z + c; } _ end _ for; d = i; } v o i d doit(dense<i32,2>& D, dense<std::co m plex<f32>,2> P) { map(mandel)(d,p); } dense<std::co m plex<f32>,2> pos; bind(pos, c _ pos, cols, rows) dense<i32,2> dest; bind(dest, c _ dest, cols, rows) call(doit)(dest, pos); Array Building Blocks ArBB ArBB ArBB C++ C++ ArBB ArBB std::complex ArBB C++ max_count call 28

MKL MKL Greg Henry Shane Story MKL MKL hotspot MKL BLAS (Basic

(VML) (VSL) (PARDISO) BLAS MKL ScaLAPACK BLAS (PBLAS) FFT IA-32/

Duo Core i7 MKL MKL OpenMP* LAPACK BLAS BLAS VML FFT FFT LAPACK

29 MKL MKL Greg Henry Shane Story MKL MKL hotspot MKL BLAS (Basic Linear Algebra Subroutines) LAPACK (Linear Algebra PACKage) (FFT) (VML) (VSL) (PARDISO) BLAS MKL ScaLAPACK BLAS (PBLAS) FFT IA-32/ 64 AMD* Linux* Windows* Mac OS* X MKL DGEMM BLAS MKL MKL Core 2 Duo Core i7 MKL MKL OpenMP* LAPACK BLAS BLAS VML FFT FFT LAPACK BLAS LAPACK MKL 29

MKL OpenMP* MKL MKL_NUM_THREADS OpenMP* MKL MKL OpenMP* MKL OpenMP* / OpenMP* OpenMP* MKL MKL Microsoft* GNU OpenMP* OpenMP* MKL_DOMAIN_NUM_THREADS mkl_domain_set_num_threads() MKL MKL 2 BLAS 4

30 MKL OpenMP* MKL MKL_NUM_THREADS OpenMP* MKL MKL OpenMP* MKL OpenMP* / OpenMP* OpenMP* MKL MKL Microsoft* GNU OpenMP* OpenMP* MKL_DOMAIN_NUM_THREADS mkl_domain_set_num_threads() MKL MKL 2 BLAS 4 MKL_ALL=2, MKL_BLAS=4 Cilk Plus TBB pthreads (Linux*) OpenMP* MKL MKL MKL_NUM_THREADS 1 MKL LAPACK BLAS LAPACK BLAS BLAS LAPACK DGETRF BLAS LAPACK (MPI) MKL MP LINPACK (DGETRF ) MPI-OpenMP* 1 1 MPI OpenMP* MPI MKL OpenMP* MPI MKL 1 OpenMP* 1 OpenMP* MKL MKL MKL_DYNAMIC FALSE TRUE mkl_set_dynamic() FFT 2 MKL Web : [BLAS] [LAPACK] [MKL] [MPI] [MPI] [OPENMP] [SCALAPACK] 30

0 30 50 % LSTC dyna 1,500 fluent 3,000 4.

31 print : Don Gunning Nick Meng Paul Besl MPI 4.0 / / print MPI % LSTC dyna 1,500 fluent 3, / ISV

32 (H) k 5k 7.5k 10k 15k 20k 1. source /opt/intel/itac/ /bin/itacvars.sh export LD _ PRELOAD=/opt/intel/itac/ /slib/libVT.so # 8p run runexec small _ model.pre -np 8 --mpi-options -trace --machines-file $PBS _ NODEFILE #256p run runexec large _ model.pre -np mpi-options -trace --machines-file $PBS _ NODEFILE / MPI trace mpi-options trace ITAC / STF P0 P1 P7 P0 P1 32

33 MPI BCAST 2. / MPI MPI : [ ] : [ ] MPI BCAST 3. / MPI MPI : : 33

34 4. / 256 : MPI : 5. / MPI MPI 3 MPI_Bcast MPI MPI MPI 6 7 MPI_BCAST 6 7 MPI MPI 34

35 6. / / : MPI : MPI / 35

MPI 4 MPI MPI 5 6 7 MPI _BCAST MPI_BCAST MPI_BCAST I_MPI_ADJUST_BCAST=4 / ISV 64 Livermore Software Technology Corporation (LSTC) LSTC OpenMP* MPI / LSTC DYNA : LSTC

36 MPI 4 MPI MPI MPI _BCAST MPI_BCAST MPI_BCAST I_MPI_ADJUST_BCAST=4 / ISV 64 Livermore Software Technology Corporation (LSTC) LSTC OpenMP* MPI / LSTC DYNA : LSTC LS-DYNA 1, LSTC DEVELOPER SPOTLIGHT & UNIX System V (ISS) IA 32 IA IA 32 IA ERP Sparc* Sun* Solaris* IA Linux* Larrabee Larrabee IA-32 Itanium Larrabee 36

37 8. / 4 9. /

38 Xeon Xeon MPP (MPI) MPI OpenMP* LSTC CYL1E6 : LSTC / print 100 MPP DYNA MPI LSTC / 8 9 MPI OpenMP* MPI MPI MPI 100 : LSTC LS DYNA MPP DYNA HYBRID LS-DYNA > OpenMP* MPI 1 OpenMP* LS-DYNA > MPI I/O 10 38

39 11. / MPI_BCAST MPI MPI_RECV 12. /

40 13. / ITAC MPI 4.0 HYBRID LS-DYNA 100 / MPI_RECV MPI_BCAST / MPI_RECV 40

41 SIMD SIMD 2 SSE2 SIMD 3 SSE3 SIMD 3 (SSSE3) # Web

インテル(R) Visual Fortran Composer XE

インテル(R) Visual Fortran Composer XE Visual Fortran Composer XE 1. 2. 3. 4. 5. Visual Studio 6. Visual Studio 7. 8. Compaq Visual Fortran 9. Visual Studio 10. 2 https://registrationcenter.intel.com/regcenter/ w_fcompxe_all_jp_2013_sp1.1.139.exe