2 3 4
(Neural Network) (Deep Learning) (Deep Learning) (
x
x
= ax + b x
x
x
? x
x
x w σ b = σ(wx + b) x w b w b
.2.8.6 σ(x) = + e x.4.2 -.2 - -5 5
x w x2 w2 σ x3 w3 b = σ(w x + w 2 x 2 + w 3 x 3 + b) x, x 2, x 3 w, w 2, w 3 b w, w 2, w 3 b
x w x2 w2 σ x3 w3 b = σ(w x + w 2 x 2 + w 3 x 3 + b) x, x 2, x 3 w, w 2, w 3 b w, w 2, w 3 b
x w x2 w2 σ x3 w3 b = σ(w T x + b) x x = x 2 x 3 w = w w 2 w 3
( ) x
(3 2 ) x x2 x3 2
(3 2 ) x x2 x3 2
(3 2 ) x x2 x3 2
(3 2 ) x x2 x3 2
(3 2 ) x x2 x3 2
(3 2 ) x x2 x3 2
x x2 x3 2, 2 [, ]
x w x2 w2 Σ x3 w3 b = w x + w 2 x 2 + w 3 x 3 + b x, x 2, x 3 w, w 2, w 3 b w, w 2, w 3 b
σ σ x σ σ x σ Σ σ σ [, ] (, )
x -2-3 5 2 6 5 u = σ(5x + 5).8 4 4 σ -3 u 2 = σ( 3x + 6) u 3 = σ( 2x + 4) = σ(2u + u 2 + 4u 3 3).6.4.2 - -5 5 x
x 5-3 -2 5 2 6 4 4 Σ -3 u = σ(5x + 5) u 2 = σ( 3x + 6) u 3 = σ( 2x + 4) = 2u + u 2 + 4u 3 3 4 3 2 - -2-3 - -5 5 x
x x x2 x3 2
= σ(wx + b) w =. b =..2.8.6.4.2 -.2 - -5 5
= σ(wx + b) w = 5. b =..2.8.6.4.2 -.2 - -5 5
= σ(wx + b) w =. b =..2.8.6.4.2 -.2 - -5 5
= σ(wx + b) w = 5. b =..2.8.6.4.2 -.2 - -5 5
= σ(wx + b) w = 5. b =..2.8.6.4.2 -.2 - -5 5
= σ(wx + b) w = 5. b = 5..2.8.6.4.2 -.2 - -5 5
= σ(wx + b) w = 5. b =..2.8.6.4.2 -.2 - -5 5
= σ(wx + b) wx + b = b/w = 2.2.8.6.4.2 -.2 - -5 5
.2.8.6.4.2 -.2 - -5 5.2 + =.2.8.6.4.2.8.6.4 (-) -.2 - -5 5.2 -.2 - -5 5
.2.8.6.4.2 -.2 - -5 5.2.8.6.4 + 7 (-7) = 9 8 7 6 5 4 3 2 - - -5 5.2 -.2 - -5 5
x 5 σ 7-5 5 σ -7 Σ = 9 8 7 6 5 4 3 2 - - -5 5
.2.8.6.4.2 -.2 - -5 5.2.8.6.4 + 4 (-4) = 9 8 7 6 5 4 3 2 - - -5 5.2 -.2 - -5 5
x 5-5 σ 4-5 σ -4 Σ = 9 8 7 6 5 4 3 2 - - -5 5
9 8 7 6 5 4 3 2 - - -5 5 + 9 8 7 6 5 4 3 2 - - -5 5 = 9 8 7 6 5 4 3 2 - - -5 5
σ x 5-5 7 5 σ -7-5 5 σ 4 5 - -4 Σ = 9 8 7 6 5 4 3 2 - - -5 5 σ
x
x x
x x + + + x x
4 3 2 - -2 (, ) [, ] -3 - -5 5 x σ = =.8.6.4.2 σ - -5 5 x
4 3 2 - -2 (, ) [, ] -3 - -5 5 x x 5-2 -3 5 2 6 4 4 Σ -3 σ = =.8.6.4.2 σ - -5 5 x
4 3 2 - -2 (, ) [, ] -3 - -5 5 x x 5-2 -3 5 2 6 4 4 Σ -3 σ = =.8.6.4.2 σ x - -5 5 x 5-2 -3 5 2 6 4 4 σ -3
= σ(a x + a 2 x 2 + c) σ(a x + a 2 x 2 c) 2.5.5 -.5 - -.5-2 -2 -.5 - -.5.5.5 2 a = r cos θ a 2 = r sin θ r = 2 θ = c =
= σ(a x + a 2 x 2 + c) σ(a x + a 2 x 2 c) 2.5.5 -.5 - -.5-2 -2 -.5 - -.5.5.5 2 a = r cos θ a 2 = r sin θ r = 2 θ = π/4 c =
= σ(a x + a 2 x 2 + c) σ(a x + a 2 x 2 c) 2.5.5 -.5 - -.5-2 -2 -.5 - -.5.5.5 2 a = r cos θ a 2 = r sin θ r = 2 θ = 2π/4 c =
= σ(a x + a 2 x 2 + c) σ(a x + a 2 x 2 c) 2.5.5 -.5 - -.5-2 -2 -.5 - -.5.5.5 2 a = r cos θ a 2 = r sin θ r = 2 θ = 3π/4 c =
N = 8 2.5.5 -.5 2 3 - -.5-2 -2 -.5 - -.5.5.5 2
N = 6 2.5.5 2 4 6 -.5 - -.5-2 -2 -.5 - -.5.5.5 2
N = 32 2.5.5 4 8 -.5 - -.5-2 -2 -.5 - -.5.5.5 2
N = 36 2.5.5 4 2 -.5 2 - -.5-2 -2 -.5 - -.5.5.5 2
b w b2 u w 2 x w 2 b3 u2 w 2 2 b w 3 u3 w 2 3 u = σ(w x + b ) u 2 = σ(w2 x + b 2 ) u 3 = σ(w3 x + b 3 ) = σ(w 2 u + w2 2 u 2 + w3 2 u 3 + b)
x b w b2 w 2 w 3 b3 u u2 u3 w 2 w 2 2 b w 2 3 + - t E t E = ( t)2 2
x b w b2 w 2 w 3 b3 u u2 u3 w 2 w 2 2 b w 2 3 + - t E E
f (x, ) (x, ) min f (x, ) (x n, n ) (x n+, n+ ) x n+ = x n α f x (x n, n ) n+ = n α f (x n, n ) α
σ(x) = + e x σ(x) = e x + e x σ (x) = ( e x ) ( + e x ) 2 = + e e x x + e x = σ(x)( σ(x))
x w σ b = σ( + wx + + b) x = σ ( ) w = σ( ) { σ( )} w = ( )w
x w σ b = σ( + wx + + b) w = σ ( ) x = ( )x b = σ ( ) = ( )
x w σ b = σ( + wx + + b) = ( )w x = ( )x w b = ( )
E w 2 = ( t) w 2 = σ( + w 2 u + ) w 2 = ( )u E w 2 = ( t)( )u
E b = ( t) b = σ( + b) b = ( ) E b = ( t)( )
E = ( t) w w w u u w = u u w = σ( + w 2 u + ) u = ( )w 2
u = σ( + w x + ) u w = u ( u )x E w = ( t)( )w 2 u ( u )x = E w 2 w 2 ( u )x
E = ( t) b b b u u = u b u b = σ( + w 2 u + ) u = ( )w 2
u = σ( + b) u b = u ( u ) E b = ( t)( )w 2 u ( u ) = E b w 2 u ( u )
E w 2 k E b = ( t)( )u k = ( t)( ) E = E w 2 wk wk 2 k ( u k )x E = E b k b w k 2 u k ( u k ) (Back Propagation)
w 2 k := w 2 k α E w 2 k b := b α E b w k := w k α E w k b k := b k α E b k (Back Propagation)
u u2 + - u3 t
u w 2 x u2 w 2 2 + - u3 w 2 3
x 2 2 t.5.2.6.9.3 5 α =.
.9.8.7.6.5.4.3.2. -2 -.5 - -.5.5.5 2
.9.8.7.6.5.4.3.2. -2 -.5 - -.5.5.5 2
.9.8.7.6.5.4.3.2. -2 -.5 - -.5.5.5 2
.9.8.7.6.5.4.3.2. -2 -.5 - -.5.5.5 2
.9.8.7.6.5.4.3.2. -2 -.5 - -.5.5.5 2
2.9.8.7.6.5.4.3.2. -2 -.5 - -.5.5.5 2
5.9.8.7.6.5.4.3.2. -2 -.5 - -.5.5.5 2
.9.8.7.6.5.4.3.2. -2 -.5 - -.5.5.5 2
x b u w b2 w 2 u2 v w 2 c w 3 w 2 2 w 2 2 b w 2 22 w 3 2 v2 c2 + - t E
x b u w b2 w 2 u2 v w 2 c w 3 w 2 2 w 2 2 b w 2 22 w 3 2 v2 c2 + - t E
x b u w b2 w 2 u2 v w 2 c w 3 w 2 2 w 2 2 b w 2 22 w 3 2 v2 c2 + - t E
x b u w b2 w 2 u2 v w 2 c w 3 w 2 2 w 2 2 b w 2 22 w 3 2 v2 c2 + - t E
8
2 3 4 5 6 7 8 9 2 77 5 3 85 8 6 4 73 5 2 5 9 6 8 5 7 9 5 2 89 8 86.5 %
(Back Propagation)