2008 : 80725872

1 2 2 3 2.1.......................................... 3 2.2....................................... 3 2.3......................................... 4 2.4 ().................................. 4 3 5 3.1........................................ 5 3.2................................. 6 4 8 4.1.................................... 8 4.2.................................. 8 4.3........................................ 9 4.4....................................... 10 5 11 5.1.............................. 11 6 12 6.1........................................ 12 6.2.................................... 12 6.3....................................... 13 6.4....................................... 13 7 14 7.1.......................................... 14 7.2.......................................... 17 8 23 9 24 1/24

1 1997 [1] *1 1 [5] *1 () 2

2 2.1 3 1 l(x), ψ a,t (x), f(x) a, t (translation) (dilation) 1 3 2.2 x y W (1), W (2) I, J, K x = {x i 1 < i < I} (1) x i = l(x i ) (2) h = W (1) x (3) ( ) h hj t j j = ψ a,t (h j ) = ψ (4) a j h = {h j 1 < j < J} (5) 3

y = W (2) h (6) y k = f(y k ) (7) y = {y k 1 < k < K} (8) 2.3 3 1. 2. 3. 1 2 3 *2 3 1 2.4 () (1) (2) a,t 2 (1) *2 0 MexicanHat ψ a,t (x) = (1 2X 2 )e X2, X = (x t)/a 5 a = 1 8 4

3 3.1 2-2-1 *3 2-2-1 XOR 10-bit tight encorders *4 (-1,1) (BP) (0.0,5.0) 5000epochs 0.07 100 () Microsoft Windows XP on Intel R Mac with Bootcamp Intel R Core T M 2 Duo CPU t7700 @ 2.40GHz, 2GB RAM Microsoft Visual Studio 2008, C++ 1 ( s) (%) Sigmoid 3.7 371 1042 100 Haar 3.4 19 45 98 MexicanHat 0.9 174 442 100 2 XOR *3 2 2 1 3 i-j-k *4 N log 2 N encorders() N-bit tight encorders 5

( s) (%) Sigmoid 4.2 110 3258 100 Haar 3.0 36 1044 11 MexicanHat 0.7 308 9717 13 3 10-bit tight encorders *5 ([1][3] *6 ) *7 () 3.2 3.2.1 1 ([1][3] ) 1 41-20-3 2-2-1 XOR *5 1 1 3 1 / *6 1 1 1 / () *7 6

3.2.2 *8 ( ) (-1,1) 1 () (encorders ) BP ( *9 ) ( * 10 ) 3.2.3 () *8 *9 (4) a j *10 (4) t j 7

4 4.1 3.2.1 1. ( W (1) ) 2. (a j, t j ) 3. ( W (2) ) 4. 4.2 4.2.1 W (1) W (1) W (1) 4.2.2 N N K-means a j t j / 1 1 0,1,2,3 4 {0,1,2,3} {0,1},{2,3} 8

2 {0},{1},{2},{3} 4 4.2.3 W (2) 0.5 0.0006 (-0.5,0.5) W (2) W (1) 4.2.4 W (2) 4.3 XOR 10-bit tight encorders 0.1 5.0 100 100 () ( s) ( s) (%) Sigmoid 3.7 371 20 1042 100 Haar 3.4 19 17 45 98 MexicanHat 0.9 174 19 442 100 4 XOR ( s) ( s) (%) Haar 2.1 5 181 29 100 MexicanHat 0.2 286 163 1189 89 5 XOR 9

( s) ( s) (%) Sigmoid 4.2 110 21 3258 100 Haar 3.0 36 26 1044 11 MexicanHat 0.7 308 21 9717 13 6 10-bit tight encorders ( s) ( s) (%) Haar 2.7 59 880 1702 27 MexicanHat - - - - 0 7 encorders 4.4 XOR Haar 7 K-means XOR MexicanHat encorders BP W (1) K-means K-means XOR encorders [0.0,1.0] W (1) [0.0,1.0] K-means 10

(1)W (1) (2) (1) ( ) (2) K-means 5 5.1 tanh Gabor MexicanHat 3 5.1.1 tanh [, ] 0 [S 0, S 1 ] 0 5.1.2 tanh (9) * 11 ψ(x)dx = 0 (9) *11 ψ(ω) 2 < ω (9) 11

5.1.3 (10) W f ψ (t, a) = 1 ( ) i t ψ f(i) (10) a a a t 1 a a i 6 1 [5]-[7] 6.1 1 [8] 6.2 2 x, y w, v θ = (w 1,..., w m, v 1,..., v n ) (11) y = f(x, θ) = v j ψ(w i x i ) (12) i,j ε y = f(x, θ) + ε (13) 12

2 [8] ε x, θ y x q(x) p(y, x, θ) = q(x) 1 2 e 1 2 (y f(x,θ))2 (14) θ 6.3 3 4 6.4 N θ E[(ˆθ θ)(ˆθ θ)] T 1 N G 1 (θ) (15) [ l(y, x, θ) l(y, x, θ) T ] G(θ) = E (16) θ θ 13

= plateau 3 [8] 4 [8] G(θ) 1 l η θ θ t+1 = θ t ηg 1 (θ t ) l(y t, x t, θ t ) θ t (17) G 1 (1) 0 (2) * 12 G 1 7 (1) (2) 2 7.1 7.1.1 3 Haar 5 MexicanHat 6 *12 tanh 14

1 0 Haar 0 * 13 * 14 5 Haar 6 MexicanHat 7.1.2 xor * 15 2 * 16 xor 2 1 1 7-8 3 1 1 2 9 2 9 1 7.1.3 [0.0, 3.0] 30 * 17 10 30 *13 *14 *15 z t+1 = f(x t, y t, z t ) z t+1 [0.1] *16 y = (x2 1 x2 2 )sin(10x 1x 2 )+1 2, x 1, x 2 [ 1.1] *17 5000 15

7 8 9 2 Amari [8] (1)1 0 (2)1 2 (1) (2) Haar MexicanHat 16

Haar (2) MexicanHat (2) 7.2 7.2.1 xor / 2.9 86.7% (26/30) 603.3 33.4 XOR 1030 11 4 500 10 11 12 1 * 18 1 1 0 2 *18 2 0-1 0 1 17

1 (1) 0 (1) 12 Haar / 2.5 3.3% (1/30) - - Haar XOR 5000 29 14 Haar [0,1] * 19 13 30 1 *19 1 18

13 14 MexicanHat 1.6 100% (30/30) 54.6 48.8 MexicanHat XOR 15 15 140 7.2.2 0.3 87.6% (26/30) 516.1 300.1 17 + 180 18 0 0/2 (2) 0 1 19

15 16 17 MexicanHat 0.3 90% (27/30) 215.3 166.8 MexicanHat 20

18 20 1000 2000 19 20 21

21 7.2.3 2-0% (0/30) - - 2 30 22 23 23 5 1 0 MexicanHat 1.1 100% (30/30) 297.2 95.4 2 MexicanHat 7.2.3 50 22

23 22 24 8 95% 23

[9] [1] Qinghua Zhang, Using Wavelet Network in Nonparametric Estimation, IEEE Transaction on Neural Networks, vol.8, Issue 2, 1997, pp.227-236 [2] Kang Li, Jian-Xun Peng, Neural input selection A fast model-based approach, Neurocomputing 70, 2007, pp.762-769 [3] Cheng-Jian Lin, Wavelet Neural Networks with a hybrid Learning Approach, JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 22, 2006, pp.1367-1387 [4] Rui Xu, Survey of Clustering Algorithms, IEEE Transaction on Neural Networks, vol.16, Issue 3, 2005, pp.645-678 [5] Shun-ichi Amari, Natural Gradient Works Efficiently in Learning, Neural Computation 10, 1998 [6] Shun-ishi Amari, Hyeyoung Park, Tomoko Ozeki, Singularities Affect Dynamics of Learning in Neuromanifolds, Neural Computation 18, 2006 [7] Florent Cousseau, Tomoko Ozeki, Shun-ichi Amari, Dynamics of Learning in Multilayer PerceptronsNear Singularities, IEEE Transactions on Neural Networks, vol.19, no.8, 2008 [8], I, //, vol.49, no.8, 2005 [9] A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang, Phoneme recognition using time-delay neural networks, IEEE Trans. Acoustics, Speech, Signal Processing, vol. 37, pp. 328 339, 1989 9 24