Vol.221-HPC-178 No.19 MPI 1,a) 1 1 1 1 Extra-P Extra-P TSUBAME3. NPB 256 A C D 19.3% 5% MPI,,, 1. Extra-P [5]Extra-P 1 1-5-1, Chofugaoka, Chofu, Tokyo 182 8585, Japan a) arima@hpc.is.uec.ac.jp Extra-P Extra-P Extra-P MPI 2. 2.1 CPU 1
[4] TAU Score-P Extra-P [3][4] 2.2 Extra-P Extra- P[5] 21 [2] Extra-P Extra-P 2.3 IF 異なる条件で複数回 解析対象のアプリケーションを実行してプロファイルを取得する 問題サイズもしくは実行プロセス数を変数としてモデルの構築を行う 構築したモデルの中で 最も適合度の高いモデルを選択する 1: 1 3. 3.1 Extra-P 1 1 1 3.2 Vol.221-HPC-178 No.19 4 x 2
y (1) y = ax + b (1) (2) y = a log 1 x + b (2) (3) y = a x + b (3) x x y (4) ax + b (x < x ) y = ax + b (x x ) (4) 1 4. 4.1 2 1: TSUBAME3. 54 CPU Memory 12.15PFlops 138,24GB 2: TSUBAME3. Intel Xeon E5-268 V4 14(28) 2.4GHz 256GB 153.6GB/s GPU NVIDIA Tesla P1 4.2 TSUBAME3. TAU TSUBAME3. TSUBAME3. 54 1 CPU(Intel Xeon E5-268 V4) 2 TSUBAME3. 1 2 TAU (Tuning and Analysis Utilities) TAU C Python [4] NAS Parallel Benchmarks NAS Parallel Benchmarks (NPB) NPB 3 5 3 [1]NPB A, B, C, D 4 B A 4 C B 4 D C 16 4.3 4.3.1 3.2 4 (MAPE) (F t ) (A t ) 5 Vol.221-HPC-178 No.19 3
Vol.221-HPC-178 No.19 IS EP CG MG FT BT SP LU 3: NAS Parallel Benchmarks 3 3 1 5 1 LU 4.3.2 MAP E = 1% N N A t F t A t (5) t=1 [%] = 1 2 1 1 128 256 2 A C D 4.3.3 () () PC 4.4 4.4.1 MAPE 4 5 MAPE NoData 4 64 A, B, C 5 B BT SP 1, 4, 16, 64 1, 2, 4, 8, 16, 32, 64 BT, SP BT SP MAPE. 1 MAPE. MAPE 4 4.4.2 4
Vol.221-HPC-178 No.19 4: 64 (MAPE [%] MAPE [%]) [%] BT 99(.,.) 1(.,.) (NoData) (NoData) CG 69(.,.) 13(21.2, 3.1) (NoData) 18(.,.) EP 1(.,.) (NoData) (NoData) (NoData) FT 57(.,.) 6(22.5, 22.6) (NoData) 37(.,.) IS 1(.,.) (NoData) (NoData) (NoData) LU 81(.,.) 19(.1,.4) (NoData) (NoData) MG 48(.,.) 4(27., 28.2) 3(1.3, 1.3) 9(.,.) SP 98(.,.) 2(.,.) (NoData) (NoData) 5: B (MAPE [%] MAPE [%]) [%] BT 78(.,.) 22(.,.5) (NoData) (NoData) CG 69(.,.) (NoData) (NoData) 31(., 12.2) EP 1(.,.) (NoData) (NoData) (NoData) FT 62(.,.) (NoData) (NoData) 38(., 88.7) IS 82(.,.) 14(14., 14.) (NoData) 4(88.7, 88.7) LU 77(.,.) 21(., 17.2) 2(.,.) (NoData) MG 72(.,.) (NoData) 14(91., 91.) 14(19.4, 19.4) SP 79(.,.) 21(.,.5) (NoData) (NoData) 2: 64 2 3 2 64 3 B 6 8 2 1 2 3 4 3: B 3 4 3.2 4 3 2 2 5
Vol.221-HPC-178 No.19 6: 4 A, B, C, D 3 A, B, C 2 A, B 1 A 7: BT SP 5 1, 4, 16, 64, 256 4 1, 4, 16, 64 3 1, 4, 16 2 1, 4 1 1 8: BT SP 9 1, 2, 4, 8, 16, 32, 64, 128, 256 8 1, 2, 4, 8, 16, 32, 64, 128 6 1, 2, 4, 8, 16, 32 4 1, 2, 4, 8 2 1, 2 1 1 MAPE 3 3 4.4.3 4 5 4 64 5 B 1.. 4 3 4: 64 5: B 3 2 8 5 2% 5 2 1 3 1 3 1 4.4.4 64 9 B 1 9 3 1 BT SP 4 BT SP 6
9: 64 [%] [%] BT 35.6 6.225 CG 1.4 3.667 EP. 7.433 FT 15.7 5.24 IS 3.7 6.35 LU 15.1 8.229 MG 52.8 5.525 SP 2.9 4.341 19.3 5.837 1: B [%] [%] BT.1 265.1 CG 1.2 449.1 EP. 669.4 FT 6.3 538.2 IS 15. 566. LU 4.2 39.8 MG 1. 451.5 SP.1 233.5 4.6 445.5 8 9 19.32% 5.83% 1 4.65% 445.5% 5. 5.1 / Extra-P Extra-P 4.65% 4.5 19.3% 5% 5.2 3.2 4 Vol.221-HPC-178 No.19 JSPS JP2H4193 [1] Bailey, D. H.: The NAS Parallel Benchmarks, RNR-94-7 (1994). [2] Calotoiu, A., Hoefler, T., Schulz, M., Shudler, S. and Wolf, F.: Insightful Automatic Performance Modeling, https://apps.fz-juelich.de/scalasca/releases/ extra-p/slides/insightfulautomaticperformance ModelingTutorialPartI.pdf. [3] Knüpfer, A., Rössel, C., an Mey, D., Biersdorff, S., Diethelm, K., Eschweiler, D., Geimer, M., Gerndt, M., Lorenz, D., Malony, A. D., Nagel, W. E., Oleynik, Y., Philippen, P., Saviankou, P., Schmidl, D., Shende, S., Tschüter, R., Wagner, M., Wesarg, B. and Wolf, F.: Score-P: A Joint Performance Measurement Run-Time Infrastructure for Periscope, Scalasca, TAU, and Vampir (211). [4] Performance Research Lab: TAU, https: //www.cs.uoregon.edu/research/tau/home.php. [5] Technical University of Darmstadt: Extra-P, https://www.scalasca.org/scalasca/software/ extra-p/download.html. 7
MPI 1,a) 1 1 1 1 MPI TSUBAME3. NAS Parallel Benchmark L1 58.7% 11.3% MPI,,, 1. 2 CPU [1] [2], [5] 1 1 1-5-1, Chofugaoka, Chofu, Tokyo 182 8585, Japan a) hasegawa@hpc.is.uec.ac.jp [3], [7] 1
2. HPC TAU[5] TAU THROTTLE TAU THROTTLE 1 [6] Extra-P [7]Extra-P MPI OpenMP 3. 3.1 1 L1 MPI L1 L1 2 ( 8,32,128 ) L1 ( 256 ) L1 L1 L1 L1 3.2 linear,inverse,log,exponentail 4 y = ax + b (1) y = a + b ( a) x (2) y = log x + b (1 < a) log a (3) y = ab x + c (1 < b, c) (4) x y L1 a, b, c 4 4 MAPE MAPE MAPE MAP E = 1% N A t F t N A t (5) t=1 A t L1 F t L1 2
1: TSUBAME3. 54 CPU() Intel Xeon E5-268 V4 Processor(Broadwell-EP, 14, 2.4GHz) 2 RAM() 256GiB (DDR4-24 32GB 8) Intel DC P35 2TB (NVMe, PCI-E 3. x4, R27/W18) Intel Omni-Path 1Gb/s 4 DDN SFA14KXE EXAScaler 2: TSUBAME3. [KB] L1 32 L1 32 L2 256 L3 35,84 N (6) linear y = ax + b (a > ) (6) x y, a, b (1) L1 4. 4.1 TSUBAME3.[4] TSUBAME3. 1 TSUB- AME 54 2 CPU CPU 14 CPU 2 TSUBAME3. CPU L1 / L2 L3 [8] NAS Parallel Benchmarks (NPB) 6 3 [9] A, B, C, D 4 B C A B 4 D C 16 8 256 FT, IS, LU 3 D 8 1 MPI 1 3: cg ep Embarassingly parallel ft 3 is lu mg L1 TAU PAPI[1] 4.2 4 L1 L1 PC 4.3 8 128 L1 256 L1 3
4: [%] (MAPE [%], MAPE [%]) linear inverse log exponential cg 17.86 (.67, 9.35) 57.14 (.57, 14.86) 1.79 (1.75, 1.75) 23.21 (.23, 1.6) ep. (-,-) 1. (., 3.24). (-, -). (-, -) ft 7.14 (1.51, 155.19) 69.39 (., 127.4) 3.6 (8.35, 15.84) 2.41 (.31, 25.26) is 7.14 (1.1, 6.92) 48.21 (.29, 9.71) 1.79 (1.27, 1.27) 42.86 (.1, 32.82) lu 9.85 (.34, 16.9) 57.58 (.77, 61.84).76 (3.17, 3.17) 31.82 (.35, 29.87) mg 2.27 (2.26, 8.11) 72.73 (.11, 1241.3). (-, -) 25. (.81, 25.96) 16,,,,,,,,,, :64, :128, :128, :128, :128, :64:128 8 256 256 L1 A C 256 D A B 2 A C 3 A D 4 D L1 5. 5.1 L1 8,16,32,64,128,256 MAPE MAPE A,B,C,D 4 1 MAPE 1.8% mg D MAPE 2% mg D comm3 ex MAPE 1,% A,B,C,D 8,16,32 32 32 64,128,256 comm3 ex MAPE 7 6 5 4 3 2 1 A B C D cg ep ft is lu mg benchmark 1: 4 linear,inverse,log,exponential MAPE 4 4 linear ft inverse inverse MAPE 1% 127% 1241% inverse inverse log MAPE exponential linear log MAPE 33% 5.2 (7) 4
average_error 2 175 15 125 1 75 5 25 :64 :128 :128 :128 :128 :64:128 :64:128:256 :128:256 cg ep ft is lu mg average_error 7 6 5 4 3 2 1 :64 :128 :128 :128 :128 :64:128 :64:128:256 :128:256 cg ep ft is lu mg 2: ( A) 4: ( C) average_error 5 4 3 2 1 :64 :128 :128 :128 :128 :64:128 :64:128:256 :128:256 cg ep ft is lu mg average_error 7 6 5 4 3 2 1 :64 :128 :128 :128 :128 :64:128 :64:128:256 :128:256 cg ep ft is lu mg 3: ( B) average error = 1 N f N f t=1 A t F t A t (7) N f A t F t A 2 B 3 C 4 D 5 8,64,128 13.1% ABCD 4 3 4 3 lu,ft,is 3 8,16,32 16,32,64 5: ( D) cg A 8,64,128 256 5 8 relative error relative error = 1 A F A (8) A F 1% relative error relative error 1% 6 2, 3, 4 A BA CA D 5
5: cg, A function name relative error[%].tau 3.8886 main 3.9389 MAIN 3.9416 makea 9.1129 sprnvc 9.8794 conj grad 2.8559 initialize mpi.3995 randlc 14.2681 icnvrt 23.1941 vecset 12.894 sparse 2.638 alloc space 4.7269 setup submatrix info.27 setup proc info 5.8198 relaitve_cost[%] 7 6 5 4 3 2 1 :64 :128 :128 :128 :128 :64:128 :64:128:256 :128:256 cg_a ep_a ft_a is_a lu_a mg_a 7: A average_error 6 5 4 3 2 1 cg ep ft is lu mg 2 3 4 profile_number 6: relaitve_cost[%] 5 4 3 2 1 :64 :128 :128 :128 :128 :64:128 :64:128:256 :128:256 cg_b ep_b ft_b is_b lu_b mg_b 8: B (9) 2 199% 3 58.7% 6 A C 3 1% A,B,C,D A 1,4,16,256 3 1,4,16 256 256 3 32,64,128 5.3 256 relative cost = 1 C p C E (9) C E C p (relative cost) 7 1 8,64,128 18.1% cg A,B ep 7 A 1 D 6
relaitve_cost[%] 5 4 3 2 1 :64 :128 :128 :128 :128 :64:128 :64:128:256 :128:256 cg_c ep_c ft_c is_c lu_c mg_c Relative_cost[%] 12 1 8 6 4 2 cg ep ft is lu mg 2 3 4 profile_number 9: C 11: 256 relaitve_cost[%] 7 6 5 4 3 2 1 :64 :128 :128 :128 :128 :64:128 :64:128:256 :128:256 cg_d ep_d ft_d is_d lu_d mg_d 1: D D D C E C p (9) 11 11 Relative cost 2 4.% 3 11.3% 2 2% 3 25% 6. 6.1 MPI L1 TSUBAME3. NPB 1.8% 8,64,128 256 13.2% 1.8 A C 3 D 58.7% D 11.3% 6.2 L1 L2 L3 MPI MPI 8 256 A D 256 E F HPC JSPS JP2H4193 7
[1] TOP 5 November 22 https://www.top5.org/lists/top5/22/11/ (Accessed on 1/28/221) [2] Knüpfer A. et al. (212) Score-P: A Joint Performance Measurement Run-Time Infrastructure for Periscope, Scalasca, TAU, and Vampir. In: Brunst H., Müller M., Nagel W., Resch M. (eds) Tools for High Performance Computing 211. Springer, Berlin, Heidelberg. [3] PMaC Performance Modeling and Charactrization https://www.sdsc.edu/pmac/researchareas/index.html (Accessed on 1/26/221) [4] TSUBAME3. https://helpdesk.t3. gsic.titech.ac.jp/manuals/handbook.ja/jobs/ (Accessed on 221/2/23) [5] S. Shende and A. D. Malony, The TAU Parallel Performance System, International Journal of High Performance Computing Applications, SAGE Publications, 2(2):287-331, Summer 26 [6] TAU throttle https://www.cs.uoregon.edu/ research/tau/docs/tutorial/ch1s5.html (Accessed on 2/7/221) [7] Extra-P https://www.scalasca.org/software/extrap/download.html (Accessed on 1/8/221) [8] Intel Xeon E5-268 V4 https://ark.intel.com/content/www/jp/ja/ark/ products/91754/intel-xeon-processor-e5-268- v4-35m-cache-2-4-ghz.html (Accessed on 1/2/221) [9] NAS Parallel Benchmarks https: //www.nas.nasa.gov/publications/npb.html#url (Accessed on 1/1/221) [1] Terpstra, D., Jagode, H., You, H., Dongarra, J. Collecting Performance Data with PAPI-C, Tools for High Performance Computing 29, Springer Berlin / Heidelberg, 3rd Parallel Tools Workshop, Dresden, Germany, pp. 157-173, 21. 8