it-ken_open.key

Size: px

Start display at page:

Download "it-ken_open.key"

さみこしの
5 years ago
Views:

pr TheConvolutional correct labelneural is written under, Alex eachkrizhevsky image, and https://www.nvidia.

2 深層学習技術の進展 ImageNet Classiﬁcation 画像認識音声認識自然言語処理機械翻訳深層学習技術はこれらの分野において特に圧倒的な強みを見せている Figure (Left) Eight ILSVRC-2010 test Deep images and the cited4: from: ``ImageNet Classiﬁcation with Networks et the al. pr TheConvolutional correct labelneural is written under, Alex eachkrizhevsky image, and with a red bar (if it happens to be in the top 5). (Right) Fiv remaining columns show the six training images that prod

11 f : R n R f : R n R

12 f Θ : R n R Θ

13 f Θ : R n R y y

14 f Θ : R n R m Θ = {W 1, b 1, W 2, b 2, } h h Wh + b g h W h b

15 ( ) h W h b

16 Θ = {W 1, b 1, W 2, b 2, } y = f Θ (x) loss(y, y) x y

19 D = {(x 1, y 1 ), (x 2, y 2 ),, (x T, y T )} B = {(x b1, y b1 ), (x b2, y b2 ),, (x bk, y bk )} G B (Θ) = 1 K K k=1 loss(y bk f Θ (x bk )) Step 1 () Θ := Θ 0 Step 2 () B Step 3 () g := G B (Θ) Step 4 () Θ := Θ αg Step 5 () Step 2

20 loss(y, y)

33 λ β α β α β j i = λ j + k B( j)\i α k j α i j = 2 tanh 1 1 tanh ( k A(i)\j 2 βk i )

34 λ β α β α β j i = λ j + k B( j)\i w k,j α k j α i j = 2 tanh 1 1 tanh ( k A(i)\j 2 βk i )

37 N x M A w y

38 Input layer Hidden layer Output layer round Index of signal component Fig. 2. Sparse signal recovery for a 6-sparse vector. (top: the original sparse signal x, bottom: the output y = Φ θ (x) from the trained neural network. n =256,m=120)

39 r t = s t + βa T (y As t ) s t+1 = η(r t ; τ),

40 x = y Ax λ x 1 r t = s t + βa T (y As t ) s t+1 = η(r t ; τ),

41 r t = Bs t + Sy s t+1 = η(r t ; τ t )

42 r t = s t + γ t W (y As t ), s t+1 = η MMSE (r t ; τ 2 t ), { y vt 2 Ast 2 2 = max Mσ2 trace(a T A) }, ϵ τ 2 t = v 2 t N (N +(γ2 t 2γ t )M)+ γ2 t σ 2 N trace(ww T ),

43 r t = s t + γ t W (y As t ), s t+1 = η MMSE (r t ; τ 2 t ), { y vt 2 Ast 2 2 = max Mσ2 trace(a T A) }, ϵ τ 2 t = v 2 t N (N +(γ2 t 2γ t )M)+ γ2 t σ 2 N trace(ww T ), 3 2 output of MMSE estimator

44 y x

45 TISTA LISTA LAMP # of params T T (N 2 + MN +1) T (NM +2) [2] M. Borgerding and P. Schniter, Onsager-corrected deep learning for sparse linear inverse problems, 2016 IEEE Global Conf. Signal and Inf. Proc. (GlobalSIP), Washington, DC, Dec. 2016, pp

46 NMSE of TISTA, LISTA and AMP; N A i,j N (0, 1/M), N = 500, M = 250, SNR = 40dB. N TISTA LISTA AMP NMSE [db] iteration

47 Three sequences of learned parameters γ t ; A i,j N (0, 1/M), N = 500, M = 250, p = 0.1, SNR = 40dB. N TISTA1 TISTA2 TISTA value iteration

48 Three sequences of learned parameters γ t ; A i,j N (0, 1/M), N = 500, M = 250, p = 0.1, SNR = 40dB. N TISTA1 TISTA2 TISTA value iteration

49 N = 500, M = 250, p = 0.1, A i,j { 1, 1}, SNR = 40 db TISTA LISTA -15 NMSE [db] iteration

51 N M H x w y

52 mulas that are based on those of ISTA: r t = s t + γ t W (y Hs t ), ( ) rt s t+1 = tanh, θ t Fig. 1. The -th layer of the TI-detector. The trainable param

53 mulas that are based on those of ISTA: r t = s t + γ t W (y Hs t ), ( ) rt s t+1 = tanh, θ t Fig. 1. The -th layer of the TI-detector. The trainable param

54 BER TI-detector(T=50) MMSE IW-SOAV(L=1,K itr =50) IW-SOAV(L=2,K itr =50) IW-SOAV(L=5,K itr =50) SNR per receive antenna(db) R. Hayakawa and K. Hayashi, Convex optimizationbased signal detection for massive overloaded MIMO systems, in IEEE Trans. Wireless Comm., vol. 16, no. 11, pp , Nov Fig. 3. BER performance for.

55 value value index t γ t θ t index t

56 y x

58 y x

59 y x

60 W 1 y + b 1 relu W i h i 1 + b i W T h T 1 + b T y α ỹ h h i h T 1 soft staircase functions f( ; S, σ 2 )

61 W 1 y + b 1 relu W i h i 1 + b i W T h T 1 + b T y α ỹ h 1... f(r; S, sigma 2 ) h i sigma 2 = 0.0 sigma 2 = 0.1 sigma 2 = h T 1 soft staircase functions f( ; S, σ 2 ) r

62 W 1 y + b 1 relu W i h i 1 + b i W T h T 1 + b T y α ỹ h h i h T 1 soft staircase functions f( ; S, σ 2 ) y

63 f(r; S, sigma 2 ) sigma 2 = 0.0 sigma 2 = 0.1 sigma 2 = 0.5 W 1 y + b 1 relu W i h i 1 + b i y h 1 h i W T h T 1 + b T α ỹ h T 1 soft staircase functions f( ; S, σ 2 ) r x y x

64 PEGReg252x504 PEGReg504x1008 Bit error rate y Max steps=25-3 Max steps=100 Max steps=500-4 No quantization SNR (db) x

67 GAN VAE NADE, Wavenet NICE, Glow

68 GPU GPU CPU ()

72 NVIDIA TESLA GPU Google Tensor Processing Unit (TPU) 出典出典

2017 (413812)

2017 (413812) Deep Learning ( NN) 2012 Google ASIC(Application Specific Integrated Circuit: IC) 10 ASIC Deep Learning TPU(Tensor Processing Unit) NN 12 20 30 Abstract Multi-layered neural network(nn) has