π-VizStudio User Manual

Size: px

Start display at page:

Download "π-VizStudio User Manual"

いとはまきい
5 years ago
Views:

1 -VizStudio User Manual

2 -CAVE -VizStudio () -CAVE -VizStudio User Manual -VizStudio -CAVE -VizStudio User Manual -VizStudio -VizStudio Manual -VizStudio -CAVE -VizStudio Admin Manual -CAVE -VizStudio Multiverse Manual VR 3D Movie Manual i

3 MPI (ifort,icc ) -lmpi mpif90 Fortran, mpicc(c), mpicxx(c++) vizfront vizcore Fortran, C, C++ vizfront Fortran $ ifort -O3 -xavx prog.f C $ icc -O3 -xavx prog.c C++ $ icpc -O3 -xavx prog.cpp OpenMP Fortran $ ifort -O3 -xavx -qopenmp prog_omp.f C ii

4 $ icc -O3 -xavx -qopenmp prog_omp.c C++ $ icpc -O3 -xavx -qopenmp prog_omp.cpp MPI Fortran $ mpif90 -O3 -xavx prog_mpi.f C $ mpicc -O3 -xavx prog_mpi.c C++ $ mpicxx -O3 -xavx prog_mpi.cpp MPI + OpenMP ( ) Fortran $ mpif90 -O3 -xavx -qopenmp prog_hyb.f C $ mpicc -O3 -xavx -qopenmp prog_hyb.c C++ $ mpicxx -O3 -xavx -qopenmp prog_hyb.cpp (Fortran,C,C++ ) vizfront qsub run_mpi.sh $ qsub run_mpi.sh 60.vizcore $ qstat iii

5 Job id Name User Time Use S Queue vizcore mpi scuser2 00:19:12 R uv-large $ qdel 60 1: run_serial.sh #!/bin/bash #PBS -N serial #PBS -q uv-test #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=2 cd ${PBS_O_WORKDIR} dplace -c1./a.out (2 ) OpenMP 12 OpenMP : run_omp.sh 2: run_omp.sh #!/bin/bash #PBS -N openmp #PBS -q uv-large #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=12 cd ${PBS_O_WORKDIR} export KMP_AFFINITY=disabled export OMP_NUM_THREADS=12 dplace -x2 -c0-11./a.out (12 ) Intel Affinity disabled OpenMP iv

6 MPI 12 MPI : run_mpi.sh 3: run_mpi.sh #!/bin/bash #PBS -N mpi #PBS -q uv-large #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=12:mpiprocs=12 cd ${PBS_O_WORKDIR} (12 ) mpiexec_mpt -np 12 dplace -s1 -c0-11./a.out 12 MPI MPI + OpenMP MPI 4 12 MPI+OpenMP : run_hybrid.sh 4: run_hybrid.sh #!/bin/bash #PBS -N hybrid #PBS -q uv-large #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=48:mpiprocs=4 (48 ) cd ${PBS_O_WORKDIR} export KMP_AFFINITY=disabled export OMP_NUM_THREADS=12 Intel Affinity disabled 12 mpiexec_mpt -np 4 omplace -nt ${OMP_NUM_THREADS} -c 0-47./a.out hybrid v

7 i ii Intel Fortran C C dplace / omplace Intel PGI OpenMP MPI MPI + OpenMP vi

8 VizStudio 1.1: -VizStudio vizfront vizcore -CAVE 1

9 1 1.2 vizcore vizfront SGI UV 300 SGI C2112-GP2 CPU Intel Xeon E v2 ( GHz 30MB LLC) *32 Intel Xeon E v3 (8 3.2GHz 20MB LLC) *2 CPU TiB (DDR3 32GiB * 512) 256 GiB(DDR4 32GiB * 8) (home) 86TB (xfs) 86TB (nfs) (data) 342TB (xfs) 342 TB (nfs) (work) 3TB (xfs) - NVIDIA Quadro K6000 * 8GPU NVIDIA Quadro K5200 * 1GPU 10Gbps 10Gbps 1.2 vizcore vizfront OS Red Hat Enterprise Linux 6.6 Red Hat Enterprise Linux 7.4 Intel Parallel Studio XE Cluster Edition, PGI Fortran/C/C++ Server for Linux Intel Parallel Studio XE Cluster Edition MPI SGI MPT (MPI Toolkit) SGI MPT (MPI Toolkit), Intel MPI Intel MKL (Math Kernel Library) Intel MKL (Math Kernel Library) GPU NVIDIA CUDA Toolkit 7.0 NVIDIA CUDA Toolkit 5.5 Altair PBS Professional 13.0 Altair PBS Professional 13.0 NICE DCV NICE DCV EnSight VR - AVS Express MPE - ParaView ParaView ImageMagick ImageMagick FFMpeg FFMpeg POV-Ray POV-Ray 2

10 vizfront -VizStudio vizfront ssh vizfront.eccse.kobe-u.ac.jp ~]$ ssh The authenticity of host vizfront.eccse.kobe-u.ac.jp ( ) can t be established. RSA key fingerprint is 24:fa:25:92:a0:57:18:89:f9:13:56:1c:c7:50:a7:4f. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added vizfront.eccse.kobe-u.ac.jp, (RSA) to the list of known hosts. Enter passphrase for key /home/wsuser/.ssh/id_rsa : Last login: Mon Aug 17 05:14: from workstation [scuser@vizfront ~]$ openssh ssh PC ~/.ssh/config 3

11 2 2.1 ssh vizfront vizfront ~]$ cat.ssh/config Host vizfront HostName vizfront.eccse.kobe-u.ac.jp Port 22 User scuser IdentityFile ~/.ssh/id_rsa ~]$ ssh vizfront Enter passphrase for key /home/sgi-remote/.ssh/id_rsa : Last login: Mon Aug 17 09:12: from workstation [scuser@vizfront ~]$ logout exit [scuser@vizfront ~]$ logout Connection to vizfront.eccse.kobe-u.ac.jp closed vizcore vizcore [scuser@vizfront ~]$ qsub -I qsub: waiting for job 13.vizcore to start qsub: job 13.vizcore ready [scuser@vizcore ~]$ logout exit [scuser@vizcore ~]$ exit logout qsub: job 13.vizcore completed 4

12 2 2.2 ~]$ vizfront ~]$ echo $SHELL /bin/bash vizcore ~]$ echo $SHELL /bin/bash vizfront ~]$ echo $LANG ja_jp.utf-8 vizcore ~]$ echo $LANG en_us.utf VizStudio 5

13 : NFS vizcore vizfront quota (soft) quota (hard) /home/${group}/${user} xfs 86TB 800GB 1TB /data/${group}/${user} xfs 342TB 16TB 20TB /work xfs 3TB vizfront vizcore /home /data NFS /home quota /data quota $(GROUP) G 5 $(USER) /home quota [viz@vizcore ~]$ /usr/sbin/xfs_quota -c quota -u viz /home Disk quotas for User viz (50002) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/xvm [ ] /home /data quota [viz@vizcore ~]$ /usr/sbin/xfs_quota -c quota -g eccse /data Disk quotas for Group eccse (10001) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/xvm [ ] /data 6

14 /home /data /data/$(group) shared shared /data/$(group)/shared /work SSD : /home /data data /data/$(group)/$(user) /home/group/user/data -> /data/group/user 2.4 PC vizfront PC scp [wsuser@workstation ~]$ scp testfile scuser@vizfront.eccse.kobe-u.ac.jp:~/data 7

15 2 2.4 vizfront PC PC scp ~]$ scp 8

16 3 -VizStudio Intel PGI GNU Intel Math Kernel Library (MKL) MPI SGI MPI Toolkit (MPT) vizfront vizcore intel PGI GNU Intel MKL ( ) SGI(HPE) MPT (MPI ) Intel MPI (MPI ) 3.1 Intel [scuser@vizfront ~]$ command [options] sourcefile [...] command Fortran77, 90, 95, 2003 ifort $ ifort [options] program.f ISO C icc $ icc [options] program.c ISO C++ icpc $ icpc [options] program.cpp [options] 3.7 Intel Intel Intel MKL Math Kernel Library 9

17 3 3.2 Fortran /opt/intel Intel Fortran / C / C++ Intel MKL /opt/intel/composerxe/samples /opt/intel/composerxe/mkl/examples vizfront vizcore Intel SGI MPT 3.2 Fortran Fortran $ ifort -O3 -xavx prog.f $ dplace./a.out OpenMP $ ifort -O3 -xavx -qopenmp -qopt-report -qopt-report-phase=openmp \ -qopt-report-file=stdout prog_omp.f $ export KMP_AFFINITY=disabled (Intel ) $ export OMP_NUM_THREADS=4 ( ) $ dplace -x2./a.out 10

18 3 3.3 C MPI $ mpif90 -O3 -xavx prog_mpi.f $ mpirun -np 4 dplace -s1./a.out MPI + OpenMP $ mpif90 -O3 -xavx -qopenmp -qopt-report -qopt-report-phase=openmp -qopt-report-file=stdout \ prog_hyb.f -lmpi $ export KMP_AFFINITY=disabled Intel $ export OMP_NUM_THREADS=4 $ mpirun -np 4 omplace -nt ${OMP_NUM_THREADS}./a.out 3.3 C C $ icc -O3 -xavx prog.c $ dplace./a.out OpenMP $ icc -O3 -xavx -qopenmp -oqpt-report -qopt-report-phase=openmp -qopt-report-file=stdout \ prog_omp.c $ export KMP_AFFINITY=disabled (Intel ) $ export OMP_NUM_THREADS=4 ( ) 11

19 3 3.4 C++ $ dplace -x2./a.out MPI $ mpicc -O3 -xavx prog_mpi.c $ mpirun -np 4 dplace -s1./a.out MPI + OpenMP $ mpicc -O3 -xavx -qopenmp -qopt-report -qopt-report-phase=openmp \ -qopt-report-file=stdout prog_hyb.c $ export KMP_AFFINITY=disabled Intel $ export OMP_NUM_THREADS=4 $ mpirun -np 4 omplace -nt ${OMP_NUM_THREADS}./a.out 3.4 C C++ $ icpc -O3 -xavx prog.cpp $ dplace./a.out (OpenMP) 12

20 3 3.5 dplace / omplace $ icpc -O3 -xavx -qopenmp -qopt-report -qopt-report-phase=openmp \ -qopt-report-file=stdout prog_omp.cpp $ export KMP_AFFINITY=disabled (Intel ) $ export OMP_NUM_THREADS=4 ( ) $ dplace -x2./a.out MPI $ mpicxx -O3 -xavx prog_mpi.cpp $ mpirun -np 4 dplace -s1./a.out MPI + OpenMP $ mpicxx -O3 -xavx -qopenmp -qopt-report -qopt-report-phase=openmp \ -qopt-report-file=stdout prog_hyb.cpp $ export KMP_AFFINITY=disabled Intel $ export OMP_NUM_THREADS=4 $ mpirun -np 4 omplace -nt ${OMP_NUM_THREADS}./a.out 3.5 dplace / omplace OS CPU dplace omplace omplace MPI + OpenMP CPU OS CPU CPU dplace omplace 13

21 3 3.5 dplace / omplace dplace omplace -c CPU CPU CPU dplace ompleace 1 CPU 2 CPU CPU ndstat vizcore ndstat vizcore vizcore CPU -c dplace omplace CPU 7 $ dplace -c7./a.out OpenMP CPU 0 7 $ dplace -x2 -c0-7./a.out MPI CPU 0 7 $ mpirun -np 8 dplace -s1 -c0-7./a.out MPI + OpenMP MPI 4 OpenMP 4 $ export OMP_NUM_THREADS=4 $ mpirun -np 4 omplace -nt ${OMP_NUM_THREADS} -c0-15./a.out 0 15 MPI 4 OpenMP 4 MPI 0, 4, 8, 12 OpenMP MPI MPI 0 OpenMP 14

22 dplace -c <cpulist> CPU cpulist ( ) 0-3 0,1,2,3 0-7:2 0,2,4,6 0-1,4-5 0,1,4,5 0-3:2,8-9 0,2,8,9 -x2 OpenMP -s1 MPI omplace -nt ${OMP_NUM_THREADS} -c <cpulist> CPU cpulist ( ) 0-N 0,1,2,3 N ( ) 1:st=2 1,3,5,7, () 0,1,1-4 0,1,1,2,3,4 (1 2 ) 0-6:st=2, 1-7:st=2 0,2,4,6,1,3,5, :bs=2+st=4 16,17,20,21,24,25,28, Intel Math Kernel Library MKL Intel Math Kernel Library MKL MKL 15

23 3 3.6 C Fortran Intel MKL BLAS BLACS LAPACK ScaLAPACK PBLAS Sparse Solver Vector Math Library (VML) Vector Statistical Library (VSL) Conventional DFTs and Cluster DFTs FFTW interface MKL MKL $ mpif90 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core test.f Intel $ mpif90 -mkl=sequential test.f BLACS ScaLAPACK MKL $ mpif90 -lmkl_scalapack_lp64 -lmkl_blacs_sgimpt_lp64 -lmkl_intel_lp64 \ -lmkl_sequential -lmkl_core test.f Intel $ mpif90 -lmkl_scalapack_lp64 -lmkl_blacs_sgimpt_lp64 -mkl=sequential test.f MKL MKL 16

24 3 3.6 $ mpif90 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 test.f Intel $ mpif90 -mkl=parallel test.f BLACS ScaLAPACK MKL $ mpif90 -lmkl_scalapack_lp64 -lmkl_blacs_sgimpt_lp64 -lmkl_intel_lp64 \ -lmkl_intel_thread -lmkl_core -liomp5 test.f -lmpi Intel $ mpif90 -lmkl_scalapack_lp64 -lmkl_blacs_sgimpt_lp64 -mkl=parallel \ test.f -lmpi $ export KMP_AFFINITY=disabled Intel $ export OMP_NUM_THREADS=4 $ dplace -x2./a.out MKL OpenMP MKL_NUM_THREADS $ export KMP_AFFINITY=disabled Intel $ export OMP_NUM_THREADS=8 OpenMP $ export MKL_NUM_THREADS=4 MKL $ dplace -x2./a.out MKL ( ) OMP_NUM_THREADS 1 MKL OMP_NUM_THREADS OpenMP 17

25 3 3.7 Intel MKL OMP_NUM_THREADS OpenMP OMP_NUM_THREADS MKL_NUM_THREADS OpenMP MKL OpenMP MKL OMP_NESTED yesmkl MPI MPI MKL OMP_NUM_THREADS 1 MKL MPI MKL OMP_NUM_THREADS MKL_NUM_THREADS 3.7 Intel Intel -O2 -msse2 SSE2 SSE SSE2 Xeon -O3 O2 -xavx AVX SSE4.2 SSSE3, SSE3, SSE2, SSE AVX O0 18

26 3 3.7 Intel -O1 -O2 -O3 -O2 -axavx -xavx -fast -ipo -O3 -no-prec-div -static -fp-model fast=2 -xhost -fast -static (SGI MPT ) -Bdynamic vec / -qopt-report=n -qopt-report-phase=vec -qopt-report-file=stdout -no-prec-div IEEE -no-prec-sqrt -qopt-report [n] n=0: n=1: n=2: n=3: -qopt-report-file=name name -qopt-report-routine=name 19

27 3 3.7 Intel name -qopt-report-phase=name name -qopt-report-help cg ipo (ipo) loop openmp OpenMP par pgo (PGO) tcollect vec all Name x -ax IA32 HOST AVX IvyBridge (Intel Xeon E7-8800v2 ) AVX 20

28 3 3.7 Intel SSE4.2 SSE4 SSSE3 SSE3 SSE2 SSE AVX SSE4.2 Nehalem-EP(Intel Xeon SSE4.2 SSE4 SSSE3, SSE3, SSE2, SSE Core SSE4.1 SSE4 SSSE3, SSE3, SSE2, SSE 45nm Core (Intel Xeon SSE3 SSSE3, SSE3, SSE2, SSE Core2 Duo (Intel Xeon ip 1 -ipo ftz -no-ftz -fltconsistency IEEE OpenMP -qopenmp OpenMP -qopt-report=n -qopt-report-phase=omp -qopt-report-file=stdout OpenMP n=0: n=1: 21

29 3 3.7 Intel n=2: MASTER, SINGLE, CRITICAL -qopenmp OpenMP parallel -qopt-report=n -qopt-report-phase=vec -qopt-report-file=stdout n=0: n=1: n=2: n=3: 2 private shared reduction n=4: 3 n=5: -parallel -par-threshold [n] n n=0: n=100: MPI -lmpi mpi SGI MPT -static static -fast -Bdynamic -lmpi SGI MPT mpif90 mpifort mpif90 -show V -help 22

30 3 3.8 PGI 3.8 PGI PGI vizcore PBS vizcore Fortran77, 90, 95, 2003 ISO C ISO C++ pgfortran pgcc pgc++ PGI Fortran / C / C++ /opt/pgi/linux86-64/2015/examples vizcore qsub -I [scuser@vizfront ~]$ qsub -I qsub: waiting for job 17.vizcore to start qsub: job 17.vizcore ready PGI [scuser@vizcore ~]$ module load pgi64/ AutoPar [scuser@vizcore ~]$ cp -r /opt/pgi/linux86-64/2015/examples/autopar. [scuser@vizcore ~]$ cd AutoPar [scuser@vizcore AutoPar]$ make NTHREADS=4 all "Test PASSED" PGI 23

31 3 3.8 PGI vizcore exit AutoPar]$ exit logout qsub: job 17.vizcore completed 24

32 VizStudio PBS Professional PBS Pro vizcore PBS Pro CPU qsub qsub -I qstat qdel 4.2 vizcore uv-pv uv-test uv-large TB 450GB 12.15TB GB 37.5GB 900GB - 1.8TB 150GB 3.6TB 2h 1h 24h - 6h 1h 24h uv-test uv-large uv-pv vizcore ParaView GPU ParaView 25

33 ( : run_serial.sh) 4.1: run_serial.sh #!/bin/bash #PBS -N serial #PBS -q uv-test #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=2 cd ${PBS_O_WORKDIR} dplace -c1./a.out (2 ) qsub [option] <JOB_SCRIPT> -N -q -o PATH -e PATH -l select=( ) -VizStudio 1 ncpus=(cpu ) mem=( ) walltime=( ) $ qsub run_serial.sh 41.vizcore 26

34 4 4.4 OpenMP 4.4 OpenMP 12 OpenMP : run_omp.sh 4.2: run_omp.sh #!/bin/bash #PBS -N openmp #PBS -q uv-large #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=12 cd ${PBS_O_WORKDIR} export KMP_AFFINITY=disabled export OMP_NUM_THREADS=12 dplace -x2 -c0-11./a.out (12 ) Intel Affinity disabled OpenMP 4.5 MPI 12 MPI : run_mpi.sh 4.3: run_mpi.sh #!/bin/bash #PBS -N mpi #PBS -q uv-large #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=12:mpiprocs=12 cd ${PBS_O_WORKDIR} (12 ) mpiexec_mpt -np 12 dplace -s1 -c0-11./a.out 12 MPI 27

35 4 4.6 MPI + OpenMP 4.6 MPI + OpenMP MPI 4 12 MPI+OpenMP : run_hybrid.sh 4.4: run_hybrid.sh #!/bin/bash #PBS -N hybrid #PBS -q uv-large #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=48:mpiprocs=4 (48 ) cd ${PBS_O_WORKDIR} export KMP_AFFINITY=disabled export OMP_NUM_THREADS=12 Intel Affinity disabled 12 mpiexec_mpt -np 4 omplace -nt ${OMP_NUM_THREADS} -c 0-47./a.out hybrid : run_mem.sh 4.5: run_mem.sh #!/bin/bash #PBS -N largemem #PBS -q uv-large #PBS -o stdout.log #PBS -e stderr.log #PBS -l select=1:ncpus=8:mem=1800gb cd ${PBS_O_WORKDIR} dplace -c7./a.out (8 1800GB) 4.7 qstat 28

36 4 4.8 $ qstat Job id Name User Time Use S Queue vizcore mpi-gentle scuser1 28:28:46 R uv-large 51.vizcore mpi-gentle scuser2 28:24:32 R uv-large 4.8 qdel $ qdel [JOBID [JOBID ]] $ qsub runmpi.sh 60.vizcore $ qstat Job id Name User Time Use S Queue vizcore mpi-gentle scuser2 00:19:12 R uv-large $ qdel 60 $ qstat 29

37 -VizStudio User Manual

Microsoft Word - uv2000manual.docx

Microsoft Word - uv2000manual.docx 大規模可視化システム UV2000 利用マニュアル名古屋大学情報基盤センター 2014/11/13 版 UV システム利用マニュアル 1 はじめに本書大規模可視化システム UV2000 利用者マニュアルは名古屋大学情報基盤センターの利用マニュアルですご不明な点やご質問がございましたら次の連絡先にお問い合わせください問い合わせ先名古屋大学情報連携統括本部情報推進部情報基盤共同利用担当