知的学習認識システム特論９.key

Size: px

Start display at page:

Download "知的学習認識システム特論９.key"

ゆゆこかたいわ
5 years ago
Views:

1 1

1960 1970 1980 1990 2000 2010 Perceptron (Rosenblatt 57) Linear Separable (Minski & Papert 68) SVM (Vapnik

9 Perceptron (Rosenblatt 57) Linear Separable (Minski & Papert 68) SVM (Vapnik 95) Neocognitron (Fukushima 80) Back Prop. (Rumelhart+ 86) Conv. net (LeCun+ 89) Deep learning (Hinton+ 06)

10 Deep Learning(深層学習)とは神経回路(ニューラルネット)モデルを用いた人工知能技術脳の働きを模した構造と学習方式深い階層構造を持つことが特徴 It s 5 Input Recognition It s 5

16 (McCulloch&Pitts 43) f ( ) x 1 w 1 u x 2 w 2 u

17 McCulloch-Pits モデルでできることモデルパラメータ {w, θ} の変更様々な論理素子 x1 w1 x2 w 2 Σ u θ u w1 w2 θ AND OR NAND

18 x1 x2 x u j = 3X w ji x i + b j f (u) i=1 j = f u j u

19 x1 x2 x3 3X u j = w ji x i + b j i=1 2 j p( j u j ) u j p( j u j ) = e u j

21 Output Input

23 NN の学習とはモデルパラメータ {w, θ} で振る舞いが変化モデルパラメータをデータから決定する学習 x1 w1 x2 w 2 Σ u θ u w1 w2 θ AND OR NAND

24 x t t {w,θ} x2 (0,1) (1,1) (0,0) (1,0)x1 1 x1 w1 x2 w2 t 1

25 x x x2 1 x1 w1 0 x2 w2 x1

26 Perceptron (Rosenblatt 57) Linear Separable (Minski & Papert 68) Boltzmann Mach. (Hinton+ 85) SVM (Vapnik 95) Neocognitron (Fukushima 80) Back Prop. (Rumelhart+ 86) Conv. net (LeCun+ 89) Deep learning (Hinton+ 06) Simple / Complex cell (Hubel & Wiesel 59) Population coding (Desimone+ 84, Tanaka+ 84)

28 0 0 : 1 1 : 0 0

29 {x n } 1 x1 n 1(x n ) 2(x n ) x2 n x3 n 3(x n ) It's "1"

30 1 2 φ2 = sgn 0 X B@ j w j j 1CA = sgn (w 0 + w w 2 2 ) φ1 w 0 + w w 2 2 = 0

31 t w x w t {xn, tn} (xn) = tn tn n w

32 (Minsk & Papert 68) φ2 φ1

33 x2 {x n } {z n } x2 z2 x1 x1 x0 z1 1 1 z0

34 {x n } 1 x1 n x2 n x3 n 1(x n ) 2(x n ) 3(x n ) It's "5"

35 {x n } 1 x1 n x2 n x3 n w20 w21 w22 1(x n ) 2(x n ) w23 3(x n ) It's "1"

36 Perceptron (Rosenblatt 57) Linear Separable (Minski & Papert 68) Boltzmann Mach. (Hinton+ 85) SVM (Vapnik 95) Neocognitron (Fukushima 80) Back Prop. (Rumelhart+ 86) Conv. net (LeCun+ 89) Deep learning (Hinton+ 06) Simple / Complex cell (Hubel & Wiesel 59) Population coding (Desimone+ 84, Tanaka+ 84)

38 x z hidden units x D w (1) MD z M w (2) KM K inputs outputs 1 x 1 x 0 z 1 w (2) 10 z 0

39 (Irie 88, Funahashi 89) {z n } z3 x1 x1 {x n } z2 x1 z1 x1 x1 x0 1 1 z0

41 t z1 z2 x1 x2 w11 (1) u w12 (1) w21 (1) u w22 (1) z1 z2 θ θ u u u u z1 z2 w1 (2) u w2 (2) u t

42 hidden units x D w (1) MD z M w (2) KM K inputs x 1 outputs 1 x 0 z 1 w (2) 10 z 0

43 (Widrow-Hoff 60) t x w 1 1 x 1 w 1 u u x 2 w 2 u x 2 w 2 u

44 E(w) = 1 2 n t n (x n ) 2 1 w (1) w (2) x1 n 1(x n ) t1 n E(w) x2 n x3 n 2(x n ) 3(x n ) t2 n t3 n w {xn } {t > 0

45 (1) 22 w11 (1) u w12 (1) w21 (1) u w22 (1) z1 z2 u u u u z1 z2 w1 (2) w2 (2) 2 u u t

46 (1) 22 w11 (1) u w12 (1) w21 (1) u w22 (1) z1 z2 u u u u z1 z2 w1 (2) w2 (2) 2 u u t

47 tk k k wkj j z i wji x {xn, tn} {w (1) ji, w (2) kj}

48 tk i k k wkj wji j z x n n k = k(x n ; w) E n (w) = 1 2 X t n k k n 2 k

49 tk k k wkj i w ji j j

50 tk k k wkj i w ji j j δ δ

51 k uk zj uj uk uj tk k δk zj δj xi xi k = 0 (u k )( k t k )

52 En(w) (Amari 67, Bottou+11,12)

53 (Le+11) (Zeiler 12) (Duchi+11) (Kingma+15)

54 (Rumelhart+ 86) (Ackle+ 85) (Cottrell+ 87) (Sejnowski & Rosenberg 87) (Gorman & Sejnowski 88) (LeCun+ 89)

55 まとめディープラーニングの要素技術は比較的枯れている単層の Perceptron のやっていることは線形分離問題の解決階層を深くすることは入力表現の変換による線形分離能力の向上が期待出来るでもチューニング難しいので SVM が主流に学習はもっと問題ターゲットが決まっているのであればコスト関数勾配法のアプローチが取れる 55

Convolutional Neural Network A Graduation Thesis of College of Engineering, Chubu University Investigation of feature extraction by Convolution

Convolutional Neural Network A Graduation Thesis of College of Engineering, Chubu University Investigation of feature extraction by Convolution Convolutional Neural Network 2014 3 A Graduation Thesis of College of Engineering, Chubu University Investigation of feature extraction by Convolutional Neural Network Fukui Hiroshi 1940 1980 [1] 90 3