ARTIFICIAL NEURAL NET WORK 43 UNIT 43 44 45 46 47 48 WIDROW-HOFF 5 53 58 58 59 59 6 6 42
Artificial Neural Net Work 953 969 98 99 GA Genetic Algorithm Heuristic Search Radial Based Widrow-Hoff AND Unit P P2 A PR i= B A = NeuralUnit( P B)... R i 43
.8.6.4.2 -.2 -.4 -.6 -.8 - - -.5.5.8.6.4.2 -.2 -.4 -.6 -.8 - - -.5.5.8.6.4.2 -.2 -.4 -.6 -.8 - - -5 5 satlins hardlims.8.6.4.2 -.2 -.4 -.6 -.8 - - -.5.5.8.6.4.2 -.2 -.4 -.6 -.8 - - -.5.5.8.6.4.2 -.2 -.4 -.6 -.8 - - -.5.5.8.6.4.2 -.2 -.4 -.6 -.8 - - -.5.5,,,satlins,, tansig - hardlim(input,b) b=.5 -.5 plot(-.2:.:.2,hardlim(-.2:.:.2,.5),'r','linewidth',3);axis([-.2.2 -.2.2]);grid.8.6.4.2 -.2 -.4 -.6 -.8-44
P() input P(2) input2 w(,2) w(,) w(,3) Sum F Neural Unit A P(3) input3 b bias a=f(w*pb) Neural Unit P W P() P = P( 2) W = [ W(,) W(,) 2 W(,) 3 ] P() 3 P() A = F [ W W W ] P (,) (,) 2 (,) 3 ( 2) b P() 3 A= F W P b ( ) 45
P() input P(2) input2 w(,)_ Sum2 Sum F b() A() bias_ Neural Unit_ b() bias2 F A(2) Neural Unit2 P(R) input3 w(s,r) Sum b(s) biass F Neural UnitS A(S) R S P() W W L W b() 2 R P( 2) W P =, W = 2 W22 L W2 M M M O M PR ( ) W W 2 L W S S SR R b( 2) b = M bs ( ) A() W W2 L W A( 2) W2 W22 L W 2 = F M M M O M AS ( ) W W 2 L W A= F( W P b ) R S S SR R P() b() P( 2) b( 2) M M PR ( ) bs ( ) W b P A 46
W W2 P() A input Sum B() F Bias Sum Sum6 B2() P() Bias3 F input2 B() Sum2 Bias Sum3 B2() Sum9 Bias4 P() F B() input3 Sum5 Bias2 Sum4 Sum B2() Bias5 A2 F2 Sum7 F2 Sum8 F2 Sum W3 B3() Sum2 Bias6 Sum4 B3() Bias7 B3() Sum5 Bias8 F3 Sum3 F3 Sum6 F3 Sum7 A3 A=F(W*P,B) A3=F3(W3*A2,B3) A2=F2(W2*A,B2) A3=F3( * (W2*F(W*P,B), ),B3) MATLAB Neural Net Toolbox P(,) A(,) P( 2, ) A( 2, ) P(:, ) = A(:, ) = M M PR (, ) AS (, ) P(,) 2 A(,) 2 P( 22, ) A( 22, ) P(:, 2) = A(:, 2) = M M PR (, 2) AS (, 2) P(, Q) A(, Q) P( 2, Q) A( 2, Q) P(:, Q) = A(:, Q) = M M PRQ (, ) ASQ (, ) 47
P(,) P(,) 2 L P(, Q) A(,) A(,) 2 L A(, Q) P( 2, ) P( 22, ) L P( 2, Q) A( 2, ) A( 22, ) L A( 2, Q) P = A = M M O M M M O M PR (, ) PR (, 2) L PRQ (, ) AS (, ) AS (, 2) L ASQ (, ) A = F( W P b) b Neural Network Toolbox Neural Unit A = F( W P, b) Rosenblatt Artificial Neural Network Neuron Neuron Rx input W xr b Sx hardlim A Sx A=hardlim(W*P,b) W b hardlim AND AND P() P(2) A P T(Target) >> P=[ ; ; ; ]' 48
P = >> T=[ ] T= P(:,) T() P(:,2) T(2) Neural Network Toolbox rands R 2 S R=2;S=; [W,B]=rands(S,R) W = -.9846 -.2332 B = -.8663 W B hardlim A A=hardlim(W*P,B); learnp [dw db]=learnp(p,a,t); learnp E=T-A %hardlim T A dw = E*P'; % W dw db = E; % E B db dw,db W,B W=WdW B=BdB E Neural Network Toolbox M trainp 49
TP=[disp_freq max_epoch]; [W,B,epochs]=trainp(W,B,P,T,TP) disp_freq max_epoch P=[ ; ; ; ]'; T=[ ]; R=2;S=; [W,B]=rands(S,R) disp_freq=; max_epoch=; TP=[disp_freq max_epoch]; [W,B,epochs]=trainp(W,B,P,T,TP) 2 Sum-Squared Network Error for 7 Epochs -2 Sum-Squared Error -4-6 -8 - -2-4 -6 2 3 4 5 6 7 Epoch 7 P(),P(2) x,y A z 3 (,) z for PP=:.: for PP2=:.: PPP(PP*,PP2*)=hardlim(W*[PP;PP2],B); end; end mesh(:.:,:.:,ppp) 5
.8.6.4.2.5.5 Hardlim Widrow-Hoff Hardlim Widrow-Hoff, W B W B Widrow-Hofff Widrow-Hoff err_goal Widrow-Hoff (purelin) W B E=T-A W B lr lr lr / ( P T P) Neural Network toolbox lr maxlinlr lr=.99*maxlinlr(p,'bias'); 5
maxlinlr = P T max [ P ] M L Widrow-Hoff E=T-A dw = E*P'; db = E; E=lr*(T-A) dw = E*P'; db = E; learnp learnwh trainwh AND Hidrow-Hoff P=[ ; ; ; ]'; T=[ ]; R=2;S=; [W,B]=rands(S,R) disp_freq=; max_epoch=; err_goal=.25; lr=.4* maxlinlr(p,'bias'); TP=[disp_freq max_epoch err_goal lr]; [W,B,epochs]=trainwh(W,B,P,T,TP) Sum-Squared Network Error for 478 Epochs err_goal.25 52
for PP=:.: for PP2=:.: PPP(PP*,PP2*)=purelin(W*[PP;PP2],B); end; end mesh(:.:,:.:,ppp); axis([ - ]) (,).5 (,),(,) -.5 -.5.5 (,) Widrow-Hoff tansig logsig purelin 53
P input Rx Sx_ A W SxR B Sx tansig Sx W2 S2xS B2 S2x purelin A2 S2x_ A=tansig(W*P,B) A2=purelin(W2*A,B2) A2=purelin( *tansig(w*p, ),B2) W,W2 B,B2 S 4 [R,Q]=size(P); S=4; [S2,Q]=size(T); [W,B]=rands(S,R); [W2,B2]=rands(S2,S); deltalin, deltatan, deltalog Widrow-Hoff E E=lr*(T-A) dw = E*P'; db = E; learnwh E=lr*D; dw = E*P'; db = E; learnbp 54
D deltalin,deltatan,deltalog AND learnbp P=[ ; ; ; ]'; T=[ ]; [R,Q]=size(P); S=6; [S2,Q]=size(T); [W,B]=rands(S,R); [W2,B2]=rands(S2,S); lr=.2; err_goal=.2; A=logsig(W*P,B);A2=purelin(W2*A,B2); E=T-A2; SSE=sumsqr(E); for epoch=:5 if(sse < err_goal) epoch=epoch-;break; end D2=deltalin(A2,E);D=deltatan(A,D2,W2); [dw,db]= learnbp(p,d,lr); [dw2,db2]=learnbp(a,d2,lr); W=WdW;B=BdB; W2=W2dW2;B2=B2dB2; A=logsig(W*P,B);A2=purelin(W2*A,B2); E=T-A2; SSE=sumsqr(E); end 55
M trainbp clear close all P=[ ; ; ; ]' T=[ ]; [R,Q]=size(P); S=5; [S2,Q]=size(T); [W,B]=rands(S,R); [W2,B2]=rands(S2,S); disp_freq=; max_epoch=8; err_goal=.; lr=.2; TP=[disp_freq max_epoch err_goal lr]; [W,B,W2,B2,epoch,TR]=trainbp(W,B,'tansig',W2,B2,'purelin',P,T,TP); Sum-Squared Network Error for 852 Epochs Sum-Squared Error - -2-3 -4 2 3 4 5 6 7 8 W,W2 B,B2 3 56
3 figure for P=:.:; for P2=:.: aaa(p*,p2*)=purelin(w2*tansig(w*[p;p2]b)b2); end end; %%%%%%% a=mesh(:.:,:.:,aaa); d=; AZ=-37.5;EL=3; pint=;ppint=; b=gca;c=gcf; %set(a,'erasemode','xor'); set(c,'windowbuttondownfcn','d=;pint=get(c,''currentpoint'');') set(c,'windowbuttonmotionfcn',... ['if(d==) ppint=get(c,''currentpoint'');',... 'diffpint=pint-ppint;pint=ppint;',... 'AZ=AZdiffpint();EL=ELdiffpint(2);view([AZ,EL]);end']) set(c,'windowbuttonupfcn','d=;').8.6.4 (,) (,),(,),(,) AND 57
xor 3. 2. 3. W B dw,db E W B dw db dw = E*P'; db = E; dw = mc*dw(-mc)*e*p'; db = mc*db(-mc)*e; mc mc= mc=.95 Neural Network Toolbox trainbpm err_goal=.; lr=.5; momentum=.95; err_ratio=.4; TP=[disp_freq max_epoch err_goal lr momentum err_ratio]; [W,B,W2,B2,epoches,TR]=trainbpm(W,B,'tansig',W2,B2,'purelin',P,T,TP); err_ratio momentum 58
Sum-Squared Error Training for 527 Epochs 2-2 -4 2 3 4 5 Epoch 2 Learning Rate - 2 3 4 5 6 Epoch Nguyen,Widrow W B tansig logsig nwtan nwlog Neural Network Toolbox initff [W,B,W2,B2] = initff(p,s,'tansig',t,'purelin'); lr M trainbpa P=[ ; ; ; ]' T=[ ]; S=6; [W,B,W2,B2] = initff(p,s,'tansig',t,'purelin'); disp_freq=; max_epoch=8; err_goal=.2; lr=.;lr_inc =.5;lr_dec =.7; err_ratio=.4; TP=[disp_freq max_epoch err_goal lr lr_inc lr_dec err_ratio]; [W,B,W2,B2,epoches,TR]=trainbpa(W,B,'tansig',W2,B2,'purelin',P,T,TP); 59
Sum-Squared Error - -2 Training for 55 Epochs 2 3 4 5 Epoch.5 Learning Rate..5 2 3 4 5 6 Epoch M trainbpx P=[ ; ; ; ]' T=[ ]; S=6; [W,B,W2,B2] = initff(p,s,'tansig',t,'purelin'); disp_freq=; max_epoch=8; err_goal=.2; lr=.;lr_inc =.5;lr_dec =.7;; err_ratio=.4; momentum =.9; TP=[disp_freq max_epoch err_goal lr lr_inc lr_dec momentum err_ratio]; [W,B,W2,B2,epoches,TR]=trainbpx(W,B,'tansig',W2,B2,'purelin',P,T,TP); 6
Sum-Squared Error - -2 Training for 7 Epochs 2 4 6 8 2 4 6 Epoch.25 Learning Rate.2.5. 5 5 2 Epoch hardlim learnp trainp Widrow-Hoff purelin learnwh tranwh Back propagation tansig/logsig learnbp trainbp purelin learnbpm trainpbm learnbpa trainbpa learnbpx trainbpx 6