Artificial Neural Net Work GA Genetic Algorithm Heuristic Search Radial Based Widrow-Hoff AND Unit P P2 A PR i= B A = NeuralUnit( P B)..

Size: px

Start display at page:

Download "Artificial Neural Net Work GA Genetic Algorithm Heuristic Search Radial Based Widrow-Hoff AND Unit P P2 A PR i= B A = NeuralUnit( P B).."

きみおしもとり
6 years ago
Views:

1 ARTIFICIAL NEURAL NET WORK 43 UNIT WIDROW-HOFF

2 Artificial Neural Net Work GA Genetic Algorithm Heuristic Search Radial Based Widrow-Hoff AND Unit P P2 A PR i= B A = NeuralUnit( P B)... R i 43

3 satlins hardlims ,,,satlins,, tansig - hardlim(input,b) b= plot(-.2:.:.2,hardlim(-.2:.:.2,.5),'r','linewidth',3);axis([ ]);grid

4 P() input P(2) input2 w(,2) w(,) w(,3) Sum F Neural Unit A P(3) input3 b bias a=f(w*pb) Neural Unit P W P() P = P( 2) W = [ W(,) W(,) 2 W(,) 3 ] P() 3 P() A = F [ W W W ] P (,) (,) 2 (,) 3 ( 2) b P() 3 A= F W P b ( ) 45

5 P() input P(2) input2 w(,)_ Sum2 Sum F b() A() bias_ Neural Unit_ b() bias2 F A(2) Neural Unit2 P(R) input3 w(s,r) Sum b(s) biass F Neural UnitS A(S) R S P() W W L W b() 2 R P( 2) W P =, W = 2 W22 L W2 M M M O M PR ( ) W W 2 L W S S SR R b( 2) b = M bs ( ) A() W W2 L W A( 2) W2 W22 L W 2 = F M M M O M AS ( ) W W 2 L W A= F( W P b ) R S S SR R P() b() P( 2) b( 2) M M PR ( ) bs ( ) W b P A 46

6 W W2 P() A input Sum B() F Bias Sum Sum6 B2() P() Bias3 F input2 B() Sum2 Bias Sum3 B2() Sum9 Bias4 P() F B() input3 Sum5 Bias2 Sum4 Sum B2() Bias5 A2 F2 Sum7 F2 Sum8 F2 Sum W3 B3() Sum2 Bias6 Sum4 B3() Bias7 B3() Sum5 Bias8 F3 Sum3 F3 Sum6 F3 Sum7 A3 A=F(W*P,B) A3=F3(W3*A2,B3) A2=F2(W2*A,B2) A3=F3( * (W2*F(W*P,B), ),B3) MATLAB Neural Net Toolbox P(,) A(,) P( 2, ) A( 2, ) P(:, ) = A(:, ) = M M PR (, ) AS (, ) P(,) 2 A(,) 2 P( 22, ) A( 22, ) P(:, 2) = A(:, 2) = M M PR (, 2) AS (, 2) P(, Q) A(, Q) P( 2, Q) A( 2, Q) P(:, Q) = A(:, Q) = M M PRQ (, ) ASQ (, ) 47

7 P(,) P(,) 2 L P(, Q) A(,) A(,) 2 L A(, Q) P( 2, ) P( 22, ) L P( 2, Q) A( 2, ) A( 22, ) L A( 2, Q) P = A = M M O M M M O M PR (, ) PR (, 2) L PRQ (, ) AS (, ) AS (, 2) L ASQ (, ) A = F( W P b) b Neural Network Toolbox Neural Unit A = F( W P, b) Rosenblatt Artificial Neural Network Neuron Neuron Rx input W xr b Sx hardlim A Sx A=hardlim(W*P,b) W b hardlim AND AND P() P(2) A P T(Target) >> P=[ ; ; ; ]' 48

8 P = >> T=[ ] T= P(:,) T() P(:,2) T(2) Neural Network Toolbox rands R 2 S R=2;S=; [W,B]=rands(S,R) W = B = W B hardlim A A=hardlim(W*P,B); learnp [dw db]=learnp(p,a,t); learnp E=T-A %hardlim T A dw = E*P'; % W dw db = E; % E B db dw,db W,B W=WdW B=BdB E Neural Network Toolbox M trainp 49

9 TP=[disp_freq max_epoch]; [W,B,epochs]=trainp(W,B,P,T,TP) disp_freq max_epoch P=[ ; ; ; ]'; T=[ ]; R=2;S=; [W,B]=rands(S,R) disp_freq=; max_epoch=; TP=[disp_freq max_epoch]; [W,B,epochs]=trainp(W,B,P,T,TP) 2 Sum-Squared Network Error for 7 Epochs -2 Sum-Squared Error Epoch 7 P(),P(2) x,y A z 3 (,) z for PP=:.: for PP2=:.: PPP(PP*,PP2*)=hardlim(W*[PP;PP2],B); end; end mesh(:.:,:.:,ppp) 5

10 Hardlim Widrow-Hoff Hardlim Widrow-Hoff, W B W B Widrow-Hofff Widrow-Hoff err_goal Widrow-Hoff (purelin) W B E=T-A W B lr lr lr / ( P T P) Neural Network toolbox lr maxlinlr lr=.99*maxlinlr(p,'bias'); 5

11 maxlinlr = P T max [ P ] M L Widrow-Hoff E=T-A dw = E*P'; db = E; E=lr*(T-A) dw = E*P'; db = E; learnp learnwh trainwh AND Hidrow-Hoff P=[ ; ; ; ]'; T=[ ]; R=2;S=; [W,B]=rands(S,R) disp_freq=; max_epoch=; err_goal=.25; lr=.4* maxlinlr(p,'bias'); TP=[disp_freq max_epoch err_goal lr]; [W,B,epochs]=trainwh(W,B,P,T,TP) Sum-Squared Network Error for 478 Epochs err_goal.25 52

12 for PP=:.: for PP2=:.: PPP(PP*,PP2*)=purelin(W*[PP;PP2],B); end; end mesh(:.:,:.:,ppp); axis([ - ]) (,).5 (,),(,) (,) Widrow-Hoff tansig logsig purelin 53

13 P input Rx Sx_ A W SxR B Sx tansig Sx W2 S2xS B2 S2x purelin A2 S2x_ A=tansig(W*P,B) A2=purelin(W2*A,B2) A2=purelin( *tansig(w*p, ),B2) W,W2 B,B2 S 4 [R,Q]=size(P); S=4; [S2,Q]=size(T); [W,B]=rands(S,R); [W2,B2]=rands(S2,S); deltalin, deltatan, deltalog Widrow-Hoff E E=lr*(T-A) dw = E*P'; db = E; learnwh E=lr*D; dw = E*P'; db = E; learnbp 54

14 D deltalin,deltatan,deltalog AND learnbp P=[ ; ; ; ]'; T=[ ]; [R,Q]=size(P); S=6; [S2,Q]=size(T); [W,B]=rands(S,R); [W2,B2]=rands(S2,S); lr=.2; err_goal=.2; A=logsig(W*P,B);A2=purelin(W2*A,B2); E=T-A2; SSE=sumsqr(E); for epoch=:5 if(sse < err_goal) epoch=epoch-;break; end D2=deltalin(A2,E);D=deltatan(A,D2,W2); [dw,db]= learnbp(p,d,lr); [dw2,db2]=learnbp(a,d2,lr); W=WdW;B=BdB; W2=W2dW2;B2=B2dB2; A=logsig(W*P,B);A2=purelin(W2*A,B2); E=T-A2; SSE=sumsqr(E); end 55

15 M trainbp clear close all P=[ ; ; ; ]' T=[ ]; [R,Q]=size(P); S=5; [S2,Q]=size(T); [W,B]=rands(S,R); [W2,B2]=rands(S2,S); disp_freq=; max_epoch=8; err_goal=.; lr=.2; TP=[disp_freq max_epoch err_goal lr]; [W,B,W2,B2,epoch,TR]=trainbp(W,B,'tansig',W2,B2,'purelin',P,T,TP); Sum-Squared Network Error for 852 Epochs Sum-Squared Error W,W2 B,B2 3 56

16 3 figure for P=:.:; for P2=:.: aaa(p*,p2*)=purelin(w2*tansig(w*[p;p2]b)b2); end end; %%%%%%% a=mesh(:.:,:.:,aaa); d=; AZ=-37.5;EL=3; pint=;ppint=; b=gca;c=gcf; %set(a,'erasemode','xor'); set(c,'windowbuttondownfcn','d=;pint=get(c,''currentpoint'');') set(c,'windowbuttonmotionfcn',... ['if(d==) ppint=get(c,''currentpoint'');',... 'diffpint=pint-ppint;pint=ppint;',... 'AZ=AZdiffpint();EL=ELdiffpint(2);view([AZ,EL]);end']) set(c,'windowbuttonupfcn','d=;') (,) (,),(,),(,) AND 57

17 xor W B dw,db E W B dw db dw = E*P'; db = E; dw = mc*dw(-mc)*e*p'; db = mc*db(-mc)*e; mc mc= mc=.95 Neural Network Toolbox trainbpm err_goal=.; lr=.5; momentum=.95; err_ratio=.4; TP=[disp_freq max_epoch err_goal lr momentum err_ratio]; [W,B,W2,B2,epoches,TR]=trainbpm(W,B,'tansig',W2,B2,'purelin',P,T,TP); err_ratio momentum 58

18 Sum-Squared Error Training for 527 Epochs Epoch 2 Learning Rate Epoch Nguyen,Widrow W B tansig logsig nwtan nwlog Neural Network Toolbox initff [W,B,W2,B2] = initff(p,s,'tansig',t,'purelin'); lr M trainbpa P=[ ; ; ; ]' T=[ ]; S=6; [W,B,W2,B2] = initff(p,s,'tansig',t,'purelin'); disp_freq=; max_epoch=8; err_goal=.2; lr=.;lr_inc =.5;lr_dec =.7; err_ratio=.4; TP=[disp_freq max_epoch err_goal lr lr_inc lr_dec err_ratio]; [W,B,W2,B2,epoches,TR]=trainbpa(W,B,'tansig',W2,B2,'purelin',P,T,TP); 59

19 Sum-Squared Error - -2 Training for 55 Epochs Epoch.5 Learning Rate Epoch M trainbpx P=[ ; ; ; ]' T=[ ]; S=6; [W,B,W2,B2] = initff(p,s,'tansig',t,'purelin'); disp_freq=; max_epoch=8; err_goal=.2; lr=.;lr_inc =.5;lr_dec =.7;; err_ratio=.4; momentum =.9; TP=[disp_freq max_epoch err_goal lr lr_inc lr_dec momentum err_ratio]; [W,B,W2,B2,epoches,TR]=trainbpx(W,B,'tansig',W2,B2,'purelin',P,T,TP); 6

20 Sum-Squared Error - -2 Training for 7 Epochs Epoch.25 Learning Rate Epoch hardlim learnp trainp Widrow-Hoff purelin learnwh tranwh Back propagation tansig/logsig learnbp trainbp purelin learnbpm trainpbm learnbpa trainbpa learnbpx trainbpx 6

知能科学：ニューラルネットワーク

知能科学：ニューラルネットワーク 2 3 4 (Neural Network) (Deep Learning) (Deep Learning) ( x x = ax + b x x x ? x x x w σ b = σ(wx + b) x w b w b .2.8.6 σ(x) = + e x.4.2 -.2 - -5 5 x w x2 w2 σ x3 w3 b = σ(w x + w 2 x 2 + w 3 x 3 + b) x,