( )

NAIST-IS-MT8511 21 2 5

( )

4 Fast ICA, NAIST-IS- MT8511, 21 2 5. i

,,, Fast ICA, ii

Organization of Concentric Receptive Fields in the Primary Visual Cortex by Using Independent Component Analysis Takayoshi Aoki Abstract Previous experimental studies showed that neurons in the primary visual cortex (V1) have four firing properties to represent visual information. Four firing properties are named luminance gabor, oriented double-opponent, concentric single-opponent, and concentric double-opponent receptive fields. Previous theoretical studies demonstrated that redundancy compressions like independent component analysis (ICA) and sparse coding can derive basic functions corresponding to luminance gabor and oriented double-opponent receptive fields from natural images. On the other hand, emergences of basic functions corresponding to two concentric receptive fields via redundancy compression of natural images have not been reported. In this study, I propose a low pass filtering function of neurons in lateral geniculate nucleus (LGN) is important to derive concentric basic functions in addition to redundancy compression. Neurons in LGN project their axons to neurons in V1. Therefore, it is plausible that LGN contributes to form receptive fields of V1. I show that the concentric double-opponent basis function can be obtained by applying the low pass filtering and Fast ICA to natural images in a computer simulation. However, the concentric single-opponent basis function cannot be not obtained. This reason is thought that the concentric doubleopponent basis function functionally includes the concentric single-opponent basis Master s Thesis, Department of Bioinformatics and Genomics, Graduate School of Information Science, Nara Institute of Science and Technology, NAIST-IS-MT8511, February 5, 21. iii

function from the view point of redundancy compression. I also clarify how shapes of basic functions depend on ones of learning images, and how color components of basic functions depend on a color distribution of learning images. Keywords: Concentric Double-Opponent, Single-Opponent, Redundancy compressions, Fast ICA, Low-pass filter iv

1. 1 1.1................. 1 1.1.1......... 1 1.1.2........ 2 1.1.3............... 4 1.1.4............... 5 1.2.......... 1 1.2.1........................ 1 1.2.2........ 11 1.2.3............. 13 1.2.4....................... 15 1.3............. 15 1.4............................... 16 2. 18 2.1.............................. 19 2.2.......................... 19 2.3.................................. 21 2.4............................... 22 2.5 PCA................. 23 2.6 PCA.................. 26 2.7 Fast ICA............................ 27 2.8...................... 28 2.9...................... 28 2.1......... 29 3. 32 3.1................ 32 v

3.2................ 32 3.3.......... 4 3.4............. 4 3.5....................... 42 3.6............. 51 4. 54 4.1......................... 54 4.2............................... 56 5. 57 58 61 A. 61 B. 62 C. 63 D. 63 E. 64 vi

1 1 2.......................... 2 3.......................... 3 4.......................... 5 5................ 7 6............ 8 7............. 9 8.......................... 12 9................. 13 1....... 14 11............. 16 12 2 18 13.................................. 2 14.......................... 22 15......... 24 16............. 34 17 22.8Hz........ 35 18 11.4Hz........ 36 19 5.7Hz......... 37 2 35.1Hz......... 38 21 11.7Hz......... 39 22............ 41 23.............. 42 24 5.7Hz............ 45 vii

25 5.7Hz............ 46 26 5.7Hz............ 47 27 5.7Hz............ 48 28 5.7Hz............ 49 29 5.7Hz............ 5 3................. 52 31..... 53 1................. 6 2...................... 15 3........................... 55 viii

1. 1.1 1.1.1 (Photon) (Retina) (Cone) ( ) ( ) ( ) ( ) ( ) ( ) (Retinal Ganglion Cells RGCs) ( 1) L M Photon S Retina Cone Retina Luminance R-G B-Y RGCs 1 (Photon) (Retina) (Cone) (Retinal Ganglion Cells RGCs) L M S (Luminance) (R-G) (B-Y) 3 1

2 1 L 558 564nm M 53 534nm S 419 42nm [5] 3 [5] S ( ) 419 42nm( ) M ( ) 53 534nm( ) L ( ) 558 564nm( ) ( 2) [2] 1.1.2 6 2

A -(L+M+S) B +(L+M+S) -M +L C -S + : Excitatory - : Inhibitory +(L+M) D E F +(L+M+S) -(L+M+S) -M +L +(L+M) -S Luminance Channel Red-Green Opponent Channel Blur-Yellow Opponent Channel 3 (A) (C) (D) (F) (A) (D) (B) (E) (C) (F) L M S + (Excitatory) (Inhibitory) (Luminance) (R-G) (B-Y) 3 ( 3) 3A D 3A 3

( 1) S 3B E 3B ( ) L M 3C F L M (L+M) [2] 1.1.3 (Lateral Geniculate nucleus LGN) [2] [9] (Primary Visual Cortex V1) 4

A B -(L+M+S) +(L+M+S) -(L+M+S) M- L+ L+ M- C D M+ L- M- L+ M+ L+ L- M- M+ L+ L+ M- M- M+ L- 4 (A) (B) (C) (D) [1] 1.1.4 4 ( 4) 4A B C D 4A C 4B D 2 [6] 2 5

1 (Single-Opponent) (Concentric Double-Opponent) (Retina) (LGN) (Primary Visual Cortex V1) (Luminance Gabor) (Oriented Double-Opponent) Luminance Gabor Single-Opponent Oriented Double-Opponent Concentric Double-Opponent Retina LGN V1 ( 1) [2] 6

ON ON OFF OFF Single-Opponent Eye Lateral geniculate nucleus ON ON ON Single-Opponent OFF OFFOFF Primary visual cortex Luminance Gabor 5 ON OFF ( 5) ( ) 7

ON OFF ON OFF Single-Opponent Eye Lateral geniculate uncleus OFF ON ON ON OFF OFF Single-Opponent M+ L- Primary visual cortex L+ M- M+ L- L+ M- Oriented Double-Opponent 6 ON OFF ( 6) 8

Single-Opponent ON ON OFF OFF ON ON ON Eye Lateral geniculate uncleus OFF OFF OFF ON OFF ON OFF OFF Single-Opponent ON OFF ON OFF OFF ON ON Primary visual cortex L+ M- M+ L- M+ L- L+ M- Concentric Double-Opponent 7 ON OFF ( 7) [3] ( ) 9

2 1 [11][18] 1.2 1.2.1 ( 1) ( ) = ( 1) ( 1) + ( 2) ( 2) +... (1) 1 ( ) Q bit/sec R bit/sec R Q Q R 1 bit/sec R Q 1

4 ( 4A) ( 4B) ( 4C) ( 4D) (Independent Component Analysis ICA) ( A) ICA ICA 1.2.2 Olshausen ( A C) [15] ( 8 ) ( 8 ) 1.1.4 [16] K( f ) 11

8 192 (16 16) [15] ( 2) ( ) 4 f K( f ) = High( f )Low( f ) = f exp (2) f f =2(cycles/picture) High( f ) Low( f ) K( f ) 2 12 f

A B K(f) 1 1 fy 1 1 1 1 f fx 9 (A) f K(f) (B)XY f x X f y Y 1 [16] 9 1.2.3 Olshausen Dharmesh ICA [21] C 1 + 1-13

図 1 先行研究において 独立成分分析で得られた基底関数 学習にはカラーの 自然画像を用い 得られた 18 個の基底関数 (6 6 3) の内半分を表示した 各画 素の色は 赤 緑 青 3 色の組み合わせより表現される 赤 緑 青はそれぞ れ実数値を持つ 黒色は赤 緑 青の全てが の値であることを示す + は正 の値を持つ領域を は負の値を持つ領域を示す つまり 1 つの基底関数を正 負に分けて表示している これらの基底関数は 初期視覚野にある白黒ガボール 型及び二重反対色ガボール型の受容野特性を持つ 図は [21] より引用した I 行 4 列内において + 行の基底関数には 輝度を示す白線がある それに隣 接して 行も白線がある つまり 白黒ガボール型の基底関数であった V 行 3 列の基底関数は 赤と緑色が存在し 赤緑型の二重反対色ガボール型の基底関 数であった IV 行 2 列において 青と黄色が存在し 青黄型の二重反対色ガボー ル型の基底関数であった 14

2 Olshausen et al.[15] Dharmesh et al.[21] This study Filter Contrast filter Low-pass filter Redundancy compressions Sparse coding ICA Fast ICA Luminance gabor Oriented double-opponent Single-opponent Concentric double-opponent 1.2.4 Olshausen Dharmesh ( 2) Dharmesh Olshausen 1.3 Elizabeth ( B) [1] ( 11A) ( ) 15

A 6 B 3 Spikes/sec 3 Spikes/sec 15.1 1 1 Spatial freq (c/deg).1 1 1 Spatial freq (c/deg) 11 (A) (B) [1] ( 11B) [4][19] 11A B 1.4 Fast ICA 2 16

Fast ICA 3 4 5 17

2. 材料と方法 本研究では google 画像検索を用いてランダムに選んだ 2 枚のカラー自然画像 を学習サンプルとして用いた (図 12) 電気生理学実験に基づいたローパスフィル ターを学習サンプルに掛けた (図 14) また アルゴリズムを簡単化するために 画 像サンプルの各画素値における平均を 分散を 1 とする標準化を行った 大きく 分散の異なる変数に対して そのまま主成分分析 (Principal Component Analysis PCA) や ICA を適用すれば その結果は分散の大きな変数の影響を強く受け 変 数間の関係を正しく得られない可能性があるからである 計算時間と 図 12 本研究において学習サンプルとして用いた 2 枚のカラー自然画像 18

PCA ( 15) PCA 1 ICA Aapo Hyvarinen Fast ICA [8][1] 2.1 ( ) 3 8bit 13 ( ) ( ) (255 255 255) m m 3 (m m ) X R X G X B X = {X R X G X B } 3 x R 1,1 x R 1,m X R =..... x R m,1 x R m,m (3) 256 256 X R 256 256 2.2 1 19

Purple Red 255 Black (,,) Blue 255 Yellow Aqua White (255,255,255) 255 Green 13 3 255 3 1mm 1,Hz 5Hz 256 256 256 256 1/128 4 3.9Hz 5Hz (1/128) = 3.9Hz (4) Hz ( ) 2

5.7Hz(3.9Hz 13) 13 22.8Hz 11.7Hz 2.3 X R X G X B 5 6 7 x r = x r E[X R ] s x g = x g E[X G ] s x b = x b E[X B ] s (5) x r x g x b ( 6) E[X] E[X] = 1 x r,g,b (6) N r N N = R + G + B s ( 7) s = 1 N g b (x r,g,b E[X]) 2 (7) r g xr x g x b 1 255 b 21

2.4 ˆx n 16 16 ( 14) 1 1 16 16 768 ˆx n 15 1 75 ˆX ˆX 768 15 A B 256 256 pixel Low-pass filter C Standardization D 16 16 pixel 1 Red Plane Green Plane Blue Plane 1 17 16 256 257 513 E E E + + 512 768 1 2 256 257 258 512 513 514 768 14 (A) 256 256 (B) (C) (D) (16 16 ) 22

2.5 PCA PCA ˆX = {ˆx n } n = 1,..., N ˆx n D PCA M < D ( 15) 1 (M = 1) D u 1 u 1 u T 1 u 1 = 1 T ˆx n u T 1 ˆx n u T 1 x x 1 N N {u T 1 ˆx n u T 1 x}2 = 1 N n=1 = 1 N = 1 N N {u T 1 (ˆx n x)} 2 n=1 N {u T 1 (ˆx n x)}{u T 1 (ˆx n x)} n=1 N {u T 1 (ˆx n x)}{(ˆx n x) T u 1 } n=1 = u T 1 Su 1 (8) S S = 1 N N (ˆx n x)(ˆx n x) T (9) u T 1 x u 1 n=1 u 1 u T 1 u 1 = 1 λ 1 ( D ) E(u 1 ) = u T 1 Su 1 + λ 1 (1 u T 1 u 1) (1) E( ) u 1 23

1 Input Data X^ 1 Basis Function 1 1 F 256 257 15, 256 257 256 257 25 256 257 512 514 512 514 512 514 512 514 768 1 PCA Reduction 15, 768 1 Eigen Value Decomposition ICA 768 ED 1/2 Restoration 1 25 768 1 M=25 PCAed Data M M=25 M Z* Independent Component s 15 (PCA) (Input Data) (PCAed Data) (ICA) (Independent Component) (ED 1/2 ) (Basis Function) Su 1 = λ 1 u 1 (11) u 1 S u T 1 ut 1 u 1 = 1 24

u T 1 Su 1 = λ 1 (12) u 1 λ 1 1 M S M λ 1,..., λ M M u 1,..., u M PCA x S S M M 1 768 D = 768 PCA 25 (M = 25) M D ˆx n = (ˆx T n u a )u a + (ˆx T n u a )u a (13) a=1 a=m+1 13 1 25 25 13 2 M z n = (ˆx T n u a )u a (14) a=1 z n 1 M M (Z = (z 1... z n ) T ) M=D=768 25

2.6 PCA Z = (z 1... z n) T E[z iz j] = δ i j (15) z i 1 I E{z z T } = I 1 PCA n ˆx Z Z = Vˆx (16) V V ˆx V = D 1/2 E T (17) D = diag(d 1 d n ) C x E = (e 1 e n ) C x = E{ˆxˆx T } 1 C x E ˆx PCA ( 9 12) E E T E = EE T = I D C x = EDE T Z E[Z Z T ] = VE[ˆxˆx T ]V T = D 1/2 E T EDE T ED 1/2 = I (18) Z Z 16 17 (ED 1/2 )Z = ˆx (19) Z ˆx 26

2.7 Fast ICA Z = (z 1... z R) T s = (s 1... s R ) T Z = As (2) A z r A s r s = A 1 Z (21) A 1 W s y = (y 1 y 2... y r ) y = WZ (22) y y r [7][17] [7][17] Fast ICA J(y) y H(y) = p(y) log p(y)dy (23) J(y) = H(y gauss ) H(y) (24) y gauss y y J(y) 27

y Z [7][17] 2.8 J(y) y y y r ( ) [14] J(y r ) 1 12 E{y3 r} 2 + 1 48 kurt(y r) 2 (25) kurt 4 kurt ( ) y r = E{y 4 r } 3 [ E{y 2 r } ] 2 = E{y 4 r } 3 (26) y r 1 4 y 3 r y4 r 2.9 3 G i E{G i (y 3 r)} 2 G 1 G 2 G 1 G 2 J(y r ) k i (E{G 1 (y r )}) 2 + k 2 (E{G 2 (y r )} E{G 2 (ν)}) 2 (27) k 1 k 2 ν 1 y r 1 27 28

G 1 (y r ) = y 3 r G 2 (y r ) = y 4 r ( 25) 2 G 2 G J(y r ) [ E{G (y r )} + E{G (µ)} ] 2 (28) G G G [8] G (y r ) = 1 log (cosh y r ) (29) a 1 28 29 [8] Fast ICA 2.1 22 W w Fast ICA w w T Z J(w T Z ) E{(w T Z ) 2 } = w 2 = 1 ω ( D ) J(w T Z ) = [ E{G ( w T Z ) } + E{G (µ)} ] 2 + ω( w 2 1) (3) w 29

J(w T Z ) w = [ E{G(w T Z )} ] 2 + 2E{G(µ)} E{G(wT Z )} + ω wwt w w w = 2E{G(w T Z )}E{Z g(w T Z )} + 2E{G(µ)}E{Z g(w T Z )} + 2ωw = 2 [ E{G(w T z)} + E{G(µ)} ] E{Z g(w T Z )} + 2ωw = 2γE{Z g(w T Z )} + 2ωw (31) w = w/ w (32) γ E{G(w T z)} + E{G(µ)} 32 w w T Z w w g G( 29) γ 33 w = E{Z g(w T Z )} (33) 33 [8] 29 α w 33 34 (1 + α)w = E{Z g(w T Z )} + αw (34) E{(w T Z ) 2 } = w 2 = 1 E{G(w T Z )} ( D ) F = (1 + α)w = E{Z g(w T Z )} + αw = (35) w α 35 ( E ) w F F w = E{Z Z T g (w T Z )} + αi (36) 3

g ( ) g( ) 36 1 E{Z Z T g (w T Z )} E{Z Z T }E{g (w T Z )} = E{g (w T Z )}I (37) F ŵ = w [ E{g (w T Z )} + α ] 1 [ E{Z g(w T Z )} + αw ] (38) 38 E{g (w T Z )} + α ˇw = E{Z g(w T Z )} E{g (w T Z )}w (39) Fast ICA 1 w PCA ED 1/2 M = 25 16 16 3 (F) ( 15) 31

3. 3.1 Dharmesh 2 ( 16A) ( 16B C) 16B 16C 2 1 16B C Dharmesh ( 1) 3.2 22.8Hz 11.7Hz ( 17 21) ( ) ( ) 17 21 17A B 22.8Hz 17A B ( 17C D) 32

18A B 11.4Hz 17B 18B ( 18C) ( 18D) 19A B 5.7Hz 19B 18B ( ) ( ) ( 19C D) ( 8 1 16 18) 2A B 35.1Hz 2B 19B ( ) ( ) ( 2C D) 21A B 11.7Hz 2B 21B ( ) ( ) ( ) ( 21C D) ( 17 21) 4.1 33

A B + C - 16 (A) (B) (C) 34

A B C + D - 17 22.8Hz (A) (C) 16 35

A B C + D - 18 11.4Hz (A) (C) 16 19 36

A B C D 19 5.7Hz (A) (C) 16 37

A B C D 2 35.1Hz (A) (C) 16 38

A B C + D - 21 11.7Hz (A) (C) 16 ( ) ( ) ( ) 39

3.3 1 22A 22B 22B 22C 22D 22B 5.7Hz 22B 22D 22B 22E 22C 22E 3.4 Fast ICA ( 23) 64 64 255 ( 23A) 1( 23B) 2( 23C) 3 B C ( ) ( 1 2) 23A C 2 ( 16 16 ) ( 23D F) ( 23D) 16 16 2 64 64 4

A B C D E 22 (A) (B) (C)B (D) 5.7Hz (E)D 2 1 ( 23E) ( 23D) 2 ( 23F) ( 23D F) ( 23A C) 41

A B C D E F 23 (A) (C) 64 64 (D)A 2 (E)B 1 2 (F)C 2 2 3.5 3.2 2 ( 24 29) 3 2 42

1 4 ( ) 3 (13) 3 ( ) 24A ( ) 3 24C ( ) 25C 26C 3 ( ) 24 26 3 27C 28C 29C 24 26 24C 25C 26C 24C 26C 26B 25C ( 24C 26C) ( 25C) 1 1 13 ( )=(255 255) (Purple) 24C 25C 26C 24C 43

24B 25C 25B 25C 26C ( 24C 25C) ( 26C) 13 ( )=(255 255 ) (Yellow) 3.2 ( ) 44

A +B -R C 5 1 4 1 -G +G Blue.5 B +R 1 4 1.2 -B -5-5 Green 5 1 4 1 Number.6 5 5 Blue -5-5 Green.5 24 5.7Hz (A) (B)2 (C)B XY 45

A +R -G C 5 1 4 1.2 -B +B Red.6 B Number +G 1 4 1.2.6 -R -5-5 Blue 5 5 Red -5-5 Blue 5 1 4 1.2.6 25 5.7Hz (A) (C) 24 46

A +G -B C 5 1 4 1 -R +R Green.5 B +B 1 4 1.2 -G -5-5 Red 5 1 4 1 Number.6 5 Green -5-5 Red 5.5 26 5.7Hz (A) (C) 24 47

A +G +R C 5 1 4 2 -B +B Green 1 B Number -R 2 1 1 4 -G -5-5 Blue 5 5 Green -5-5 Blue 5 1 4 2 1 27 5.7Hz (A) (C) 24 48

A B Number -G 1 4 +B -R +R -B +G Blue C 5-5 -5 Red 2 1 5 5 Blue -5-5 Red 1 4 2 1 5 1 4 2 1 28 5.7Hz (A) (C) 24 49

A B Number -B 1 4 +R -G +G -R +B Red C 5-5 -5 Green 2 1 5 5 Red -5-5 Green 1 4 2 1 5 1 4 2 1 29 5.7Hz (A) (C) 24 5

3.6 ( ) ( 3) 1 ( ) ( ) 3.2 3A (1 2 3) =( ) 6 3B 3 3B 6 3 1 ( ) 2 ( ) 3 ( ) 1 24 29 2 (1 2 3) =( ) 3.5 ( 31) 3 ( 31A ) ( 31A C D E F G ) 31A 24C 24C 31A 31A 24C 3B (Aqua) (13) 1 1 51

A 3 1 2 B Axis (1, 2, 3) Basis Function Axis (1, 2, 3) Basis Function (R, G, B) (B, R, G) (R, B, G) (G, R, B) (B, G, R) (G, B, R) 3 (A) (1 2 3) (R G B) 1 2 3 (1 2 3) =(R G B) ( 13 ) (B) 31B ( 13) 52

Blue A 1 4 B 5 1.2 5.6 Red 1 4 1.5-5 -5 C 5 5 Green 1 4 1.2-5 -5 D 5 5 Blue 1 4 2 Green.6 Blue 1-5 -5 E 5 Green 5 Red 1 4 2 1-5 -5 F 5 Red 5 Red 1 4 2 1-5 -5 5 Blue -5-5 Green 5 31 R G B (1 2 3)=( ) (1 2 3)=( ) (A) (B) (C) (D) (E) (F) 53

4. 4.1 Fast ICA ( 18 2) ( 22) ( 23) ( 21) ( 24 29) ( 3 31) PCA ( 14 15) ( ) PCA ED 1/2 18 2 3 ( 16 21) 2 2 1 54

3 ( ) (Layer) [1] Layer Single-Opponent (n = 13) Double-Opponent (n = 51) 4/13, 31% 19/51, 37% /13, % 1/51, 2% 1/13, 8% 9/51, 18% /13, % 3/51, 6% 1/13, 8% 2/51, 4% 4/13, 31% 6/51, 12% 3/13, 23% 11/51, 22% ( 3) ( ) 1 4 ( ) 3 7 [12] 55

4.2 1 [1] ( ) ( 3) 2 1 ( ) [2] 56

5. ( ) ( ) Fast ICA 57

58

[1] http://www.cs.helsinki.fi/u/ahyvarin/. [2] 2., 1998. [3] Bevil R. Conway and Margaret S. Livingstone. Spatial and temporal properties of cone signals in alert macaque primary visual cortex. J. Neurosci., 26(42):1826 1846, 26. [4] A. M. Derrington and P. Lennie. Spatial and temporal contrast sensitivities of neurones in lateral geniculate nucleus of macaque. J. Physiol, 357:219 24, 1984. [5] Karl R. Gegenfurtner and Daniel C. Kiper. Color vision. Annu. Rev. Neurosci., 26:181 26, 23. [6] Hubel D. H. and Wiesel T. N. Receptive fields, binocular interaction and functional architecture in the cat s visual cortex. J. Physiol, 195:215 244, 1968. [7] Aapo Hyvarinen. Survey on independent component analysis. Neural Computing Surveys, 2:94 128, 1999a. [8] Aapo Hyvarinen. Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. on Neural Networks, 1(3):626 634, 1999b. [9] L. Iwai and H. Kawasaki. Molecular development of the lateral geniculate nucleus in the absence of retinal waves during the time of retinal axon eye-specific segregation. Neuroscience, 159:1326 1337, 29. [1] Elizabeth N. Johnson, Michael J. Hawken, and Robert Shapley. The orientation selectivity of color-responsive neurons in macaque v1. J. Neurosci., 28. [11] Avi Karni and Dov Sagi. Where practice makes perfect in texture discrimination: Evidence for primary visual cortex plasticity. Neurobiology, 88:4966 497, 1991. 59

[12] Carole E. Landisman and Daniel Y. Color processing in macaque striate cortex: Relationships to ocular dominance, cytochrome oxidase, and orientation. J. Neurophysiol, 87:3126 3137, 22. [13] D. Luenberger. Optimization by Vector Space Methods, Wiley, 1969. [14] Jone M. and Sibson R. What is projection pursuit? J. of the Royal Statistical Society, ser. A, 15:1 36, 1987. [15] Bruno A. Olshausen and David J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381:67 69, 1996. [16] Bruno A. Olshausen and David J. Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision Res., 37(23):3311 3325, 1997. [17] Comon P. Independent component analysis - a new concept? Signal Processing, 36:287 314, 1994. [18] D. R. Peeples and D. Y. Teller. Colour vision and brightness discrimination in two-month-old human infants. Science, 189:112 113, 1975. [19] Shapley R and Lennie P. Spatial frequency analysis in the visual system. Rev. Neurosci., 8:547 583, 1985. [2] E. Sernagor, S. Eglen, B. Harris, and R. Wong. Retinal development. 26. [21] Dharmesh R. Tailor, Leif H. Finkel, and Gershon Buchsbaum. Color-opponent receptive fields derived from independent component analysis of natural images. Vision Research, 4:2671 2676, 2. 6

A. I(a, b) a b I(a, b) a b 16 16 a b 1 16 1 16 16 a i I(a, b) = a i ϕ i (a, b) (4) i ϕ i (a, b) i 192 n n Î(a, b) = a i ϕ i (a, b) (41) I(a, b) Î(a, b) [ E 1 = ] 2 n I(a, b) Î(a, b) = I(a, b) a i ϕ i (a, b) a,b a,b i i E 1 ϕ i (a, b) a i a i 2 (42) 61

a i 42 E 2 n E 2 = I(a, b) a i ϕ i (a, b) a,b i 2 + λ n i [ log 1 + ( ai ) 2 ] σ σ 43 1 2 a i λ E 2 ϕ i (a, b) a i ϕ i (a, b) E 2 a i (43) E 2 = ϕ i (a, b)i(a, b) ϕ i (a, b)ϕ j (a, b)a j λ 1 + a i σ 2 a i a,b j a,b 1 + ( ) a 2 (44) i σ ϕ i (a, b) ϕ i (a, b) ϕ i (a, b) = η [ a ] i I(a, b) Î(a, b) (45) η < > a i 44 1 ϕ i (a, b) 45 1 4, B. Elizabeth 112 62

1 ( ) ( 11A) ( 11B) C. Olshausen 512 512 16 16 4, 16 16 192 [15] Dharmesh 24 32 6 6 2, ICA 6 6 18 [21] D. y k J(y k )( 28) min J(y k ), subject to K i (w) =, (i = 1,..., p) (46) 63

K i (w) = y k L(y k, λ 1,..., λ k ) = J(y k ) + k λ i H i (y k ) (47) λ 1,..., λ p (47) ( 46) L(y k, λ 1,..., λ k ) y k λ i L(y k, λ 1,..., λ k ) λ i K i (y k ) K i (y k ) = L(w, λ 1,..., λ k ) w J(w) w + k 48 46 i=1 i=1 λ i H i (w) w = (48) E. ( ) 1 1 2 2 J(w) (2.7 ) J(w) w [ ] T J(w) J(w ) = J(w) + (w w) + 1 w 2 (w w) T 2 J(w) (w w) +... (49) w 2 J(w) w J(w) w w = w 64

[ ] T J(w) J(w ) J(w) = w + 1 2 J(w) w 2 wt w (5) w 2 w 5 w (J(w ) J(w)) w = J(w) w + 1 2 2 J(w) w + w 2 [ ] = J(w) w + 1 2 = J(w) w 2 2 J(w) w w 2 ( ) 2 T J(w) w w 2 + 2 J(w) w 2 w (51) 2 J(w) w 2 51 [ ] 2 1 J(w) J(w) w = w 2 w (52) J(w) w [ ] w 2 1 J(w) J(w) = w w 2 w (53) w 53 [13] 2 1 Fast ICA ICA 65

1 66