CUDA-BLAS GPU 1,3,a) 1 2,3 2,3, GPU CUDA-BLAS,., GPU Eigen-G, MAGMA CUDA- BLAS., CUDA-BLAS ASPE.K2 dsymv, MAGMA+ASPE.K2. 1. GPU BLAS ([1], [2] ), CUDA [3] CUBLAS[4] MAGMABLAS[5]., ( ) ((SY HE)MV) HPC-138[6], HPC-146[7]. (SY HE)MV.. y := αa U or L x + βy where A(= A ) K n n, x K n, K = R or C. (1), SYMV, SYMV. GPU GPGPU, SYMV. 1 RIKE Advanced Institute for Computational Science, Kobe, Hyogo 2 Japan Atomic Energy Agency, Kashiwa, Chiba 3 CREST CREST JST, Kawaguchi, Saitama a) imamura.toshiyuki@riken.jp. GPU MAMGA[5] magma dsyevd magma dsyevdx 2stage, Eigen-G[8]., GPU, CPU GPU. Level2, Level3 BLAS GPU.,, CUDA-BLAS., MAGMA Eigen-G CPU, CPU GPU., GPU CUDA-BLAS CPU, CPU+GPU., CUDA-BLAS GPU. 2. CUDA-BLAS 2.1 CUBLAS CUBLAS[4] VIDIA CUDA SDK[3] BLAS CUDA. VIDIA Level1 Level3 ),., CUDA 1
BLAS., DGEMM GPU, CUDA., Tesla Kc (DGEMM) 1TFLOPS. 2.2 MAGMABLAS MAGMABLAS GPU MAGMA [5] BLAS. 1.6.1. MAGMABLAS CUBLAS, ([9] ), CUDA CUDA.. 2.3 KBLAS KBLAS[], [11] KAUST CUDA BLAS. Level2 GEMV SYMV, SYMV., 1.2(1.3-beta )., MAGMA, KBLAS. 2.4 ASPE.K2 ASPE.K2[1] CUDA- BLAS., GEMV, SYMV ([6], [7] ). SYMV. 2.5, EM Photonics CULA[16] CULABLAS., BLAS, CUBLAS CUBLAS )., GLAS[14],. GLAS Sørenssen GPUlab DTU Level1, Level2. [2] GEMV ( ) CUDA- BLAS. CUBLAS, MAGMA, KBLAS, ASPE.K2. 3. CUDA CUDA GPU 3. MAGMA Eigen-G. 3.1 CULA CULA[16] LAPACK CUDA. QR, + syev, syevx.., MAGMA ASPE.K2. QR,. CULA [17] CPU,.,. 3.2 MAGMA MAGMA[5] 2 magma dsyevd magma dsyevdx 2stage. magma dsyevd LAPACK dsyevd, ( 1 ) (magma dsytrd) ( 2 ) (magma dstedx) ( 3 ) (magma dormtr)., 1) (magma dsytrd) dsymv dsyr2k GPU. 2) (magma dstedx), dgemm GPU. 3) (magma dormtr), WY CPU,, dgemm GPU., magma dsyevdx 2stage,,. 2
5. ( 1 ) (magma dsytrd sy2sb) ( 2 ) (magma dsytrd sb2st) ( 3 ) (magma dstedx) ( 4 ) (magma dbuldge back) ( 5 ) (magma dormqr 2stages) magma dsytrd sy2sb, 1. dgemm, dsymm, dsyr2k GPU., Level3 BLAS. GPU Level2 Level3. sy2sb. magma dsytrd sb2st. magma dbuldge back. magma dormqr 2stages API. MAGMA GPU, CPU. 3.3 Eigen-G Eigen-G[8] EigenK, EigenExa[18] GPU. [8]. Eigen-G, magma dsyevd, ( 1 ) ( 2 ) ( 3 ) 3. CPU GPU, Eigen-G CPU GPU dgemm. 13 9 MAGMA1.4., Eigen-G magma dsyevd 2/3., ( 1 ) DSYMV. MAGMA CUBLAS, ASPE.K2 3. ( 2 ) async. 4. Eigen-G, GPU,. 4.1 CUDA-BLAS MAGMA 1.6.1 SYMV, SYMV CUBLAS MAGMABLAS., CUBLAS., CUDA-BLAS GPU. 1 2 dgemm, dsymv CUDA-BLAS. CUBLAS dgemm. GTX9, MAGMABLAS (sgemm)., dsymv ASPE.K2 CUBLAS Atomic KBLAS., CUBLAS Atomic KBLAS.,, KBLAS Atomic AtomicAdd., HPC-146 mutex ASPE.K2., ASPE.K2 dsymv. dsymv 1, 2, 3., ASPE.K2., ASPE.K2, Lower( ), Upper( ) AS- PE.K2., ASPE.K2., (dsytrd ) 1, dsymv., dsymv,., CUDA-BLAS dgemm CUBLAS dsymv ASPE.K2., dsyr2k dgemm. 3
1 DGEMM ( GFLOPS) 88 2112 3136 41 5184 68 7232 8256 Kc CUBLAS 6.5 878. 14.45 13.47 24.86 32.9.63 41.29 42.92 MAGMABLAS 1.6.1 538.65 575.48 577.6 575.13 571.95 565.84 563.17 563.88 MKL 121.32 23.87 142. 146.54 147.87 145.89 145.57 145.44 GTX9 CUBLAS 7. 125.16 137.61 142.58 143.62 143.69 143. 143.91 143.87 MAGMABLAS 1.6.1 121.34 126.19 141.74 142.75 142.67 142.89 143.2 142.96 MKL 47.18 21.77 66.8 65.72.91 64.71 65.72 69.71 2 DSYMV ( GFLOPS) 88 2112 3136 41 5184 68 7232 8256 Kc CUBLAS 6.5 9.67 13.63 15.59 16.72 17.38 17. 18.26 18.48 (Atomic) 17.56 34.22 46.62 55.4 56.23 56.94 59.21 59.54 MAGMABLAS 1.6.1 13.32 26.57 37.56.91 43.63 46. 48.91.97 MKL.57 15.31 7.77 8.4 7.91 7.97 7.76 7.24 KBLAS 1.2 23.25 43.15 49.81 54.39 56.44 57.66 58.38 59.26 ASPE.K2 1.5p2 26.13 46.71 54.88 58.82 59.21 61.73 62.39 62.64 GTX9 CUBLAS 7. 16.35 26.47.13.91 33.29 34.1 34.83 35.47 (Atomic).8.74 54.51 64.97 73.4 76.64 78.26 78.9 MAGMABLAS 1.6.1 17.81.46 39.4 44.66 47.53 49.64 51.29 53.22 MKL 4.46 4. 3.98 4. 4.3 4.15 3.85 4.8 KBLAS 1.2 23.75 48.53 63.16 73.13 77.62 79.27 79.75 79.93 ASPE.K2 1.5p3 31.72 56.18 65.94 73.13 77.33 79.47 82.19 81.53 4
3 Tesla Kc ( ) 88 2122 3136 41 MAGMA (1) trd.15.49 1.5 1.94 (2) ed.4.14.31.52 (3) tbk.1.7.14.28 Eigen-G (1) trd.11.33.74 1.42 (2) ed.4.15.24.65 (3) tbk.2.6.15.29 DSYMV (GeForce GTX 9) 4 GTX9 ( ) 88 2122 3136 41 MAGMA (1) trd.13.49 1.15 2.22 (2) ed.8.24.58 1. (3) tbk.3.18.56 1.22 Eigen-G (1) trd.11.38.86 1.69 (2) ed.6.28.73 1.39 (3) tbk.4.18.52 1.12 4.2 MAGMA Eigen-G,. 2 (magma dsyevdx 2stage), magma dsyevd. 3, 4., 88 41. Tesla Kc CPU GPU GTX9 ( Tesla Kc GTX9 ). 3 dgemm,, (1) Eigen-G, (2) MAGMA. (3). (1), dsymv. dgemm. (2) MAGMA. Eigen-G dgemm. (3), MAGMA CPU dgemm. Eigen-G CPU, GTX9 dgemm GPU 1GFLOPS, CPU 2GFLOPS 3:2. 2 ASPE.K2 1.5p3x DSYMV, Upper ASPE.K2 1.5p3x DSYMV, Lower DSYMV (GeForce GTX 9) CUBLAS 7.RC, DSYMV, Upper (atomics mode) CUBLAS 7.RC, DSYMV, Lower (atomics mode) DSYMV (GeForce GTX 9) MAGMA 1.6.1, DSYMV, Upper MAGMA 1.6.1, DSYMV, Lower MAGMA 1.6.1, DSYMV, Upper (work) MAGMA 1.6.1, DSYMV, Lower (work) DSYMV (GeForce GTX 9) KBLAS 1.2, DSYMV, Upper KBLAS 1.2, DSYMV, Lower GeForce GTX9 SYMV ( 32 ) 5
5 ( ) GPU CPU / Tesla Kc GTX9 GPU ame GK1 GM4 Compute Capability 3.5 5.2 GPU Clock (MHz) 6(boost A) 1126(boost 1216) Multiprocessors 13 16 CUDA Cores 2496 (=13*192) 48 (=16*128) Memory Capacity (MByte) 51 (GDDR5) 96 (GDDR5) Memory Clock (MHz) 5(3bit) 12(256bit) Memory Bandwidth (GB/s) 8 224 ECC Support Enabled A (ECC on ) PCI bus PCIe 2. 16 PCIe 3. 16 (host PCIe2 ) Host (a) (b) Host (a) Host (b) CPU AMD FX-81 Intel Core i7-39k CPU Core 8 6 (4FPUs) (AVX available) CPU Clock (GHz) 3.1 3.2 Memory Capacity (GB) 16 16 Linux Kernel version 3.6.11-4 3.11.- CUDA Version 7.RC 6.5 Driver Version 346.29 343.19 GU gcc Version 4.6.3 4.7.2 Intel MKL Version 13..1 13..1 6
DSYMV (GeForce GTX 9) ASPE.K2 1.5p3x DSYMV, Upper ASPE.K2 1.5p3x DSYMV, Lower KBLAS 1.2, DSYMV, Upper KBLAS 1.2, DSYMV, Lower MAGMA 1.6.1, DSYMV, Upper MAGMA 1.6.1, DSYMV, Lower MAGMA 1.6.1, DSYMV, Upper (work) MAGMA 1.6.1, DSYMV, Lower (work) CUBLAS 7.RC, DSYMV, Upper (atomics mode) CUBLAS 7.RC, DSYMV, Lower (atomics mode) DSYMV (Tesla Kc) ASPE.K2 1.5p2 DSYMV, Upper ASPE.K2 1.5p2 DSYMV, Lower KBLAS 1.2, DSYMV, Upper KBLAS 1.2, DSYMV, Lower MAGMA 1.6.1, DSYMV, Upper MAGMA 1.6.1, DSYMV, Lower MAGMA 1.6.1, DSYMV, Upper (work) MAGMA 1.6.1, DSYMV, Lower (work) CUBLAS 6.5, DSYMV, Upper (atomics mode) CUBLAS 6.5, DSYMV, Lower (atomics mode) 1 DSYMV CUDA-BLAS (GTX9( ), Tesla Kc( ), 32 ) 7
DSYMV (Tesla Kc) ASPE.K2 1.5p2 DSYMV, Upper ASPE.K2 1.5p2 DSYMV, Lower DSYMV (Tesla Kc) CUBLAS 6.5, DSYMV, Upper (atomics mode) CUBLAS 6.5, DSYMV, Lower (atomics mode) DSYMV (Tesla Kc) 6 Tesla Kc MAGMA+ASPE.K2 (1)trd ( ) 88 2122 3136 41 MAGMA+ASPE.K2.11.33.77 1.46 MAGMA only.15.49 1.5 1.94 Eigen-G.11.33.74 1.42 4.3 MAGMA+ASPE.K2, MAGMA+ASPE.K2. magma dsyevd (1) magma dsytrd) dsymv, ASPE.K2 dsymv ((2)(3) (1) ). MAGMA+ASPE.K2 Eigen-G., CUDA-BLAS. MAGMA Eigen-G., MAGMA BLAS GPU., MAGMA Eigen-G.,. 3 MAGMA 1.6.1, DSYMV, Upper MAGMA 1.6.1, DSYMV, Lower MAGMA 1.6.1, DSYMV, Upper (work) MAGMA 1.6.1, DSYMV, Lower (work) DSYMV (Tesla Kc) KBLAS 1.2, DSYMV, Upper KBLAS 1.2, DSYMV, Lower Tesla Kc SYMV ( 32 ) 4.4, 2 magma dsyevdx 2stage. 2., 1., MAGMA MAGMA+ASPE.K2..,. 5., CUDA-BLAS,. Eigen-G, MAGMA, CUDA-BLAS. 8
, MAGMA+ASPE.K2, CPU+single GPU. MAGMA 2stage, MAGMA.., ( : 223) ( (COE) ). [1] Imamura, T., ASPE-K2: Automatic-tuning and Stabilization for the Performance of CUDA BLAS Level 2 Kernels, 15th SIAM Conference on Parallel Processing for Scientific Computing (PP12), http://www.siam.org/meetings/pp12/ [2], Kepler GPU SGEMV, GTC Japan 14. [3] VIDIA Corporation, CUDA C Programming guide, http://docs.nvidia.com/cuda/pdf/cuda C Programm ing Guide.pdf (14). [4] VIDIA Corporation, The VIDIA CUDA Basic Linear Algebra Subroutines, http://developer.nvidia.com/cublas [5] Innovative Computing Laboratory, University of Tennessee, Matrix Algebra on GPU and Multicore Architectures, http://icl.cs.utk.edu/magma [6],,,,, Fermi, Kepler GPU SYMV,, HPC, Vol. 12-HPC-138, o. 8 (12) 1 7. [7],,,, CUDA-xSYMV,, HPC, Vol. 14-HPC-146, o. 14 (14) 1 12. [8] Imamura, T., Yamada, S., Machida, M., Eigen-G: GPUbased eigenvalue solver for real-symmetric dense matrices, th International Conference on Parallel Processing and Applied Mathematics (PPAM14), LCS8384, pp. 673-682, 14 [9] ath, R., Tomov, S., Dong, T. T., and Dongarra, J., Optimizing Symmetric Dense Matrix-vector Multiplication on GPUs, in Proceedings of 11 International Conference for High Performance Computing, etworking, Storage and Analysis, SC 11 (11) 6:1 6:. [] Abdelfattah, A., Keyes, D., and Ltaief, H., KAUST BLAS (KBLAS), http://cec.kaust.edu.sa/pages/kblas.aspx [11] Abdelfattah, A., Keyes, D., and Ltaief, H., KBLAS: High Performance Level-2 BLAS on Multi-GPU Systems, http://ondemand.gputechconf.com/gtc/14/poster /pdf/p4168 KBLAS GPU computing optimization.pdf, GTC14 (14). [12] Sørensen, H. H. B., Auto-tuning Dense Vector and Matrix-Vector Operations for Fermi GPUs, Parallel Processing and Applied Mathematics, LCS 73 (12) 619 629. [13] Sørensen, H. H. B.. Auto-Tuning of Level 1 and Level 2 BLAS for GPUs, Concurrency Computat.: Pract. Exper., Wiley (12) 1183 1198. [14] GPUlab: GLAS library version..2, http://gpulab.imm.dtu.dk/docs/ glas v..2 C cuda 4. linux.tar.gz [15] Imamura, T., Yamada, S., and Machida, M., A High Performance SYMV Kernel on a Fermi-core GPU, High Performance Computing for Computational Science VECPAR 12, LCS 7851 (13) 59 7. [16] Humphrey, J.R., Price, D. K., Spagnoli, D. K., Paolini, A. L., Kelmelis, E. J., CULA: Hybrid GPU Accelerated Linear Algebra Routines, SPIE Defense and Security Symposium (DSS), April,. [17] CULA. http://www.culatools.com/dense/ [18] EigenExa : http://www.aics.riken.jp/labs/lpnctrt/eigenexa.html EigenK http://ccse.jaea.go.jp/ja/download/eigenk.html 9