2002 avidemux MPEG-4 : : : G99P045-1

Size: px

Start display at page:

Download "2002 avidemux MPEG-4 : : : G99P045-1"

らむすすむ
5 years ago
Views:

1 2002 avidemux MPEG-4 : : : G99P045-1

2 MPEG-4 MPEG-4 PC MPEG-4 MPI XviD MPEG-4 MPEG avidemux MPI MPEG-4 PE

3 MPEG MPEG PE Myrinet OS SCore avidemux avidemux 29 i

4 Pass ii

5 MPEG-4 VOP Folon-III/Folon-IV SCore avidemux iii

6 2.1 MPEG FOLON-III FOLON-IV MPI DivX 640x frames to DivX CQ Avi 320x frames to DivX CQ iv

7 1 1.1 MPEG-4 MPEG-4 PC MPEG-4 PC Cluster Folon3 Folon4 MPEG-4 avidemux MPI(Message Passing Interface) RWCP(: Real World Computing Partnership) SCore MPEG-4 avidemux 3 PC Cluster folon3 folon4 4 avidemux 5 avidemux 6 1

8 2 avidemux MPEG-4 MPE MPEG-4 MPEG Moving Picture Expert Group ISO( ) MPEG MPEG MPEG-4 MPEG-4 / MPEG-4 MPEG-1/2 MPEG-4 2.1: MPEG MPEG-1 384kbps 4Mbps VideoCD MPEG-2 1Mbps 100Mbps DVD MPEG-4 10Mbps MPEG-7 - EPG 2

9 I B B P I I P : 2.1: (AC/DC 8x8 ) ( ) ( ) MPEG-4 VO(Video Object) VO VOP(Video Object Plane) VOP 4 I-VOP( VOP) P-VOP( VOP) B-VOP( VOP) S-VOP( VOP) VOP I-VOP VOP P-VOP I P-VOP VOP B-BOP I P-VOP I P-VOP VOP S-VOP VOP 3

10 2.2 MPEG-4 API ( ) MPEG-4 XviD API 1. API PE PE 1 1PE ( ) 4

11 2.2: MPEG-4 VOP 2.4 MPEG-4 GOV(Group Of Video Object Plane) I,B,P,S-VOP MPEG-4 GOV 2-Pass ( 1-Pass 2-Pass ) I-VOP I-VOP MPEG-4 5

12 3 PC FOLON-III FOLON-IV PE FOLON-III 9 CPU PC Myrinet BASE-TX Ethernet Pentium-III 1GHz 1152MB FOLON-IV 17 CPU PC FOLON-III Myrinet BASE- TX Ethernet Myrinet2000 FOLON-III serial FOLON-IV fiber CPU Pentium-III-S 1.26GHz 4096MB MB FOLON-III Myrinet Myrinet Myricom Gigabit LAN Ethernet Myrinet 1. Ethernet 6

13 3.1: FOLON-III CPU L2 chache main memory PE0 roquefort Pentium-III 1GHz 256kB 1152MB PE1 roquefort01 Pentium-III 1GHz 256kB 1152MB PE2 roquefort02 Pentium-III 1GHz 256kB 1152MB PE3 roquefort03 Pentium-III 1GHz 256kB 1152MB PE4 roquefort04 Pentium-III 1GHz 256kB 1152MB PE5 roquefort05 Pentium-III 1GHz 256kB 1152MB PE6 roquefort06 Pentium-III 1GHz 256kB 1152MB PE7 roquefort07 Pentium-III 1GHz 256kB 1152MB PE8 roquefort08 Pentium-III 1GHz 512kB 1152MB 3.2: FOLON-IV CPU L2 chache main memory comte Pentium-III 1.26GHz 512kB 4096MB PE0, PE1 comte00 Pentium-III 1.26GHz 512kB 2048MB PE2, PE3 comte01 Pentium-III 1.26GHz 512kB 2048MB PE4, PE5 comte02 Pentium-III 1.26GHz 512kB 2048MB PE6, PE7 comte03 Pentium-III 1.26GHz 512kB 2048MB PE8, PE9 comte04 Pentium-III 1.26GHz 512kB 2048MB PE10, PE11 comte05 Pentium-III 1.26GHz 512kB 2048MB PE12, PE13 comte06 Pentium-III 1.26GHz 512kB 2048MB PE14, PE15 comte07 Pentium-III 1.26GHz 512kB 2048MB PE16, PE17 comte08 Pentium-III 1.26GHz 512kB 2048MB PE18, PE19 comte09 Pentium-III 1.26GHz 512kB 2048MB PE20, PE21 comte10 Pentium-III 1.26GHz 512kB 2048MB PE22, PE23 comte11 Pentium-III 1.26GHz 512kB 2048MB PE24, PE25 comte12 Pentium-III 1.26GHz 512kB 2048MB PE26, PE27 comte13 Pentium-III 1.26GHz 512kB 2048MB PE28, PE29 comte14 Pentium-III 1.26GHz 512kB 2048MB PE30, PE31 comte15 Pentium-III 1.26GHz 512kB 2048MB 7

14 backborn network NIS Server, NFS Server, Router comte15 Ethernet Hub Myrinet Switch comte00 comte folon-iv Ethernet Hub roquefort07 Myrinet Switch roquefort folon-iii 3.1: Folon-III/Folon-IV 8

15 Myrinet 16 Myrinet Myrinet 2. NIC Myrinet NIC LANai LANai NIC NIC NIC NIC NIC NIC 4 3. Gigabit Ethernet 1Gbps Myrinet Gbps 4Gbps OS FOLON Redhat Linux7.3 SCore5.2 SCore SCore 5.2 Score (RCWP) PC PC SCore SCore 9

16 3.2: SCore PMv2 PMv2 PMv2 Myrinet Ethernet UDP Shmem SCore-D SCore-D Linux PM SCore-D SCore-D SCore-D MPICH-SCore MPICH-SCore MPI MPICH PMv2 MPI MPICH-SCore Version 1.0 Version 2.0 MPICH MPICH MPICH-SCore Version

17 MPICH-SCore Version 1.0 Version 1.0 PVM-SCore PVM SCore PVM3.4 PVM PVM-SCore SCore PVM SCACH SCACH PMv2 PBS MRJ ( : ) NASA PBS(Protable Batch System) SCore MPC++ C++ C++ / Omni OpenMP OpenMP API RWCP Omni OpenMP SCASH OpenMP TACO C++ TACO 11

18 4 avidemux 4.1 PE PE GUI PE PE avidemux pre MPI MPI MPI C++ mpic++ mpic++ [ options ] filename... mpic++ MPICH-SCore MPI PATH 4.2 configure make./configure CC=mpicc CXX=mpic++ 12

19 mpicc mpic++ mpif77 mpif90 C MPI C++ MPI Fortran77 MPI Fortran90 4.1: MPI /usr/bin/ld: BFD assertion fail elf-strtab.c:262 make SCore checkpointing SCore checkpointing static avidemux static configure./configure CC="mpicc -nostatic" CXX="mpic++ -nostatic" source= oss_out.cpp object= oss_out.o libtool=no \ depfile=.deps/oss_out.po tmpdepfile=.deps/oss_out.tpo \ depmode=gcc3 /bin/sh../../admin/depcomp \ mpic++ -DHAVE_CONFIG_H -I. -I. -I../.. -I/usr/include/kde/ar ( ) -fno-check-new -c -o oss_out.o test -f oss_out.cpp echo./ oss_out.cpp mv: cannot stat.deps/oss_out.tpo : ar cru libadm_audiodevice.a ADM_deviceArts.o ADM_audiodevice.o ADM_deviceoss.o oss_out.o ar: oss_out.o: gcc 2.96 dependency style gcc3 configure Makefile avidemux/adm audiodevice/ MPI configure avidemux/adm audiodevice/makefile 13

20 CC = gcc CPP = gcc -E CXX = g++ CXXXPP = g++ -E 4.3 avidemux GTK+ Gtk-WARNING **: cannot open display: localhost:10.0 ssh X Forwarding ssh ( :4.3) DISPLAY scout scout ( :4.3) MPI C++ #include <mpi++.h> C #include <mpi.h> include MPI PE PE main.cpp myid( ),nprocs( PE ) 14

21 ssh login user s console cluster host DISPLAY=localhost:10.0 cluster host DISPLAY=localhost:10.0 "scout -g pcc" environment variables copied cluster host DISPLAY=localhost:10.0 X Forwarding user s console cluster host "scrun avidemux" DISPLAY=localhost:10.0 cluster host avidemux DISPLAY=localhost:10.0 cluster host avidemux DISPLAY=localhost: : 15

22 ssh login user s console cluster host DISPLAY=user s console:10.0 "scout -g pcc" cluster host DISPLAY=user s console:10.0 environment variables cop cluster host DISPLAY=user s console:10.0 user s console cluster host "scrun avidemux" DISPLAY=user s console:10.0 cluster host avidemux DISPLAY=user s console:10.0 cluster host avidemux DISPLAY=use s console: : 16

23 int myid,nprocs; int main(int argc, char *argv[]) { extern int myid; // extern int nprocs; // PE MPI MPI Init MPI PE main MPI Init MPI Comm rank MPI Comm size int main(int argc, char *argv[]) { pthread_t tid; FCT_VOID *thread; MPI_Init(&argc,&argv); MPI_Comm_rank(MPI_COMM_WORLD,&myid); MPI_Comm_size(MPI_COMM_WORLD,&nprocs); dll GTK+ gtk init PE avidemux GTK+ atexit 17

24 atexit(onexit); gtk_set_locale(); gtk_init(&argc, &argv); gdk_rgb_init(); PE gtk main PE PE children main PE gtk widget show PE0 window1 if(myid==0) { gtk_widget_show(window1); gtk_main(); } else { children_main(); } return 0; } PE GUI avidemux/gtk gui.cpp void HandleAction(Action action) (avidemux/gui action.hxx enum ) switch PE action MPI Isend PE GUI 18

25 void HandleAction(Action action) { uint32_t nf=0; mybcast_pe0(null, 0, MPI_CHAR, (int)action, MPI_COMM_WORLD); switch(action) { case ACT_LoadWork: GUI_FileSelRead("Select workbench to load ", A_loadWorkbench); updateloaded(); return; break; case ACT_VideoConfigure: printf("\n **\n"); videocodecconfigure(); return; break; PE PE switch gtk main children main HandleAction event switcher void children_main(void) { int eventtag; printf("node %d stand by ok.\n",myid); while(1){ eventtag = MPI_ANY_TAG; mybcast_children(null, 0, MPI_CHAR, &eventtag, MPI_COMM_WORLD); event_switcher(eventtag); } } 19

26 void event_switcher(int eventtag) { uint32_t nf=0; switch(eventtag) { case : exit(0); case ACT_LoadWork: recvfile(eventtag); updateloaded(); return; case ACT_VideoConfigure: videocodecconfigure(); return; event switcher HandleAction case GUI FileSelRead recvfile mybcast pe0 mybcast children MPI MPI Isend, MPI Irecv PE0 void mybcast_pe0(void *buff, int count, MPI_Datatype datatype, int tag, MPI_Comm MPI_Request mpirequest; MPI_Status mpistatus; int dest = 1; while(1) { if(dest > nprocs -1) break; MPI_Isend(buff, count, datatype, dest, tag, comm, &mpirequest); MPI_Wait(&mpirequest, &mpistatus); dest = dest * 2; } } 20

27 void mybcast_children(void *buff, int buffsize, MPI_Datatype datatype, int *retu MPI_Request mpirequest; MPI_Status mpistatus; int tag, count, distance, dest; tag = *returntag; MPI_Irecv(buff, buffsize, datatype, MPI_ANY_SOURCE, tag, comm, &mpirequest); MPI_Wait(&mpirequest, &mpistatus); if(tag== MPI_ANY_TAG) { *returntag=mpistatus.mpi_tag; } distance = myid-mpistatus.mpi_source; MPI_Get_count(&mpistatus, datatype, &count); while(1) { distance = distance * 2; dest = myid + distance; if(dest > nprocs -1) break; MPI_Isend(buff, count, datatype, dest, *returntag, comm, &mpirequest); MPI_Wait(&mpirequest, &mpistatus); } } atexit atexit onexit PE PE

28 void onexit( void ) { MPI_Request mpirequest; MPI_Status mpistatus; if(myid==0) mybcast_pe0(null,0,mpi_char,444444,mpi_comm_world); filtercleanup(); printf("node %d exiting...\n",myid); MPI_Finalize(); } PE exit(0) onexit PE PE PE PE PE PE PE PE PE avidemux case ACT_LoadWork: GUI_FileSelRead("Select workbench to load ", A_loadWorkbench); GUI FileSelRead GUI FileSelRead A loadworkbench GUI FileSelRead avidemux/adm tookkit/filesel.cpp start thread PE 22

29 void start_thread(void *ptr) { ( ) fd=fopen(selected_file,"rb"); if(!fd) { GUI_Alert("Cannot open this file!"); /* PE */ filesize = -1; mybcast_pe0(&filesize, 1, MPI_INT, 0, MPI_COMM_WORLD); return; } else { fseek(fd, 0, SEEK_END); filesize = ftell(fd); fseek(fd, 0, SEEK_SET); buff = (char *)malloc(filesize); fread(buff, filesize, 1, fd); // mybcast_pe0(&filesize, 1, MPI_INT, 0, MPI_COMM_WORLD); // mybcast_pe0(buff, filesize, MPI_CHAR, 1, MPI_COMM_WORLD); free(buff); } fclose(fd); _callback(selected_file); PE GUI FileSelRead recvfile recvfile PE switch GUI FileSelRead 23

30 void recvfile(int action) { ( ) mpitag = 0; // mybcast_children(&buffsize,1,mpi_int,&mpitag,mpi_comm_world); // if(buffsize < 0) return; buff = (char *)malloc(buffsize); mpitag = 1; // mybcast_children(buff,buffsize,mpi_char,&mpitag,mpi_comm_world); ( ) fd = fopen(tmpfilename, "wb"); if(!fd) { printf("node: %d can t make the tmpfile\n", myid); return; } if(!(fwrite(buff, buffsize, 1, fd))) { return; } free(buff); fclose(fd); switch(action) { case ACT_LoadWork: A_loadWorkbench(tmpfilename); break; ( ) } return; } Save AVI PE PE 24

31 avidemux/gui savenew.cpp A SaveAudioNVideo PE 1PE = ( - ) / PE PE = + (1PE x ) PE = PE + 1PE - 1 PE PE PE PE PE PE PE PE if(myid==0)filegather_recvfiles(); else filegather_sendfile(); PE filegather sendfile 1. PE ( PE ) 2. PE 25

32 void filegather_recvfiles(void) { ( ) for(i = 1; i<nprocs; i++) { // MPI_Recv(&buffsize,1,MPI_INT,MPI_ANY_SOURCE,0,MPI_COMM_WORLD,&mpistatus); source = mpistatus.mpi_source; ( ) buff = (char *)malloc(buffsize); // MPI_Recv(buff,buffsize,MPI_CHAR,source,1,MPI_COMM_WORLD,&mpistatus); ofstream out(tmpsaveavifilename, ios::out ios::binary); out.write(buff,buffsize); out.close(); free(buff); } } PE filegather recvfiles 1. PE ( PE ) 2. PE 3. PE 1 26

33 void filegather_sendfile(void) { ( ) ifstream in(tmpsaveavifilename, ios::in ios::binary); in.seekg(0,ios::end); filesize = in.tellg(); in.seekg(0,ios::beg); buff = (char *)malloc(filesize); in.read(buff,filesize); MPI_Send(&filesize,1,MPI_INT,0,0,MPI_COMM_WORLD); // MPI_Send(buff,filesize,MPI_CHAR,0,1,MPI_COMM_WORLD); // free(buff); in.close(); } MPI Gather PE PE avidemux A openavi(avi ) A appendavi( AVI ) A SaveAudioNVideo( audioprocessmode videoprocessmode 0 AVI ) 27

34 A_openAvi_pe0(tmpsaveavifilename00); for(i=1;i<nprocs;i++) { tmpsaveavifilename[filename_length] = 0 + (int)(i / 10); tmpsaveavifilename[filename_length+1] = 0 +(i % 10); tmpsaveavifilename[filename_length+2] = \0 ; A_appendAvi_pe0(tmpsaveavifilename); } tmpsaveavifilename[filename_length] = \0 ; framestart = tmpframestart; frameend = tmpframeend; A_SaveAudioNVideo_pe0(realname); 28

35 5 avidemux avidemux 640x360, DivX 320x240, 900 AVI DivX CQ( ) 4 3 folon4 SMP 1 1PE 5.1 PE PE 1 / PE 5.1 sec total bcasting merging gathering PE0 enctime max PE enctime min PE enctime pe0 PE0 enctime avg PE waittime max PE PE0 PE DivX PE PE avidemux Avi PE 29

36 PE total bcasting merging gathering enctime max enctime min PE enctime pe0 enctime avg waittime max : DivX 640x frames to DivX CQ4 30

37 PE total bcasting merging gathering enctime max enctime min PE enctime pe0 enctime avg waittime max : Avi 320x frames to DivX CQ4 31

38 5.1: avidemux 32

39 6 6.1 PE PE MPEG PE PE MPEG-1/2/4 GOP(GOV) 1/ PE I/O I/O MPEG AVI I/O mmap HDD 33

40 Pass 2-Pass PE 1-Pass 1-Pass PE 1-Pass PE GUI GUI PE 34

41 Folon avidemux mean 35

42 [1] P., :MPI,,2001. [2],,,,, : Linux,, [3] PC Cluster Consortium, 2001, [4] : GIMP,, [5] : MPI GIMP core,, [6] avidemux, 2002, [7] MPEG Home Page, 2003, [8] ISO/IEC JTC1/SC29/WG11 N2564 Overview of the MPEG-4 Standard, [9] PIONEER R&D, 2002, 36

para02-2.dvi

para02-2.dvi 2002 2 2002 4 23 : MPI MPI 1 MPI MPI(Message Passing Interface) MPI UNIX Windows Machintosh OS, MPI 2 1 1 2 2.1 1 1 1 1 1 1 Fig. 1 A B C F Fig. 2 A B F Fig. 1 1 1 Fig. 2 2.2 Fig. 3 1 . Fig. 4 Fig. 3 Fig.