IPSJ SIG Technical Report Vol.2013-MUS-100 No /9/1 1,a) [1 9] 1 3 MIDI MIDI (Interonset interval) 1 NTT NTT Communication Science Labo

Size: px

Start display at page:

Download "IPSJ SIG Technical Report Vol.2013-MUS-100 No /9/1 1,a) [1 9] 1 3 MIDI MIDI (Interonset interval) 1 NTT NTT Communication Science Labo"

うたろうあきます
4 years ago
Views:

1 1,a) [1 9] 1 3 MIDI MIDI (Interonset nterval) 1 N N Communcaton Scence Laboratores, Atsug, Kanagawa , Japan he Insttute of Statstcal Mathematcs, achkawa, okyo a) ohsh.yasunor@lab.ntt.co.jp 1 Fg. 1 Vocal volume contours of excerpts from a song sung by three sngers (two professonals and one amateur) 1 [10] [11 14] A B c 013 Informaton Processng Socety of Japan 1

2 Fg. Vocal volume contours of four songs sung by a snger C A B [15 17] 1 B 4 [18] MIDI [19] (F0) [0, 1] 1 (HMM) 5 HMM HMM [] [3] D = {x t, y t } t=1 x t+1 y t+1 [0, 1]. [4] [5, 6] [] [] HMM 1 x t = [ ( ),, ] (1) t 10 ms c 013 Informaton Processng Socety of Japan

3 p(y y, X, x ) = N (y ; µ, σ ) (4) µ = k (K + η I) 1 y σ = k(x, x ) k (K + η I) 1 k Fg. 3 3 Generatve process of vocal volume contour based on mxture of Gaussan process experts N w(t) h(t) [8] y t = N/ τ= N/ ( (w(t + τ) h(τ)) ) /N () N 51 (3 ms) h(t) [3,7] x y y = [y 1,..., y ] p(y) = GP(y; 0, K + η I) = N (y; 0, K + η I) (3) GP K K,j = k(x, x j ) k(x, x j ) η I x y X x k y y ([ ]) ([ ] [ ]) y y K + η I k p = N ; 0, k k(x, x ) y y y y. 3 [3, 7] [3] p({x t, y t } t=1 Θ, Ω) (5) = ( R ) p({z t } t=1 Ω) p(y r X r, Θ, Ω)p(X r Θ, Ω) Z z t t r X r {x t : z t = r} y r {y t : z t = r} R Θ Ω (5) Z R z t ( 1 ) R {z t } t=1 ( ) r {z t : z t = r} r θ x r = {µ r, Σ r } c 013 Informaton Processng Socety of Japan 3

Fg. 4 4 Graphcal representaton of proposed model 5 Fg.

{θr x } R Ω) (6) R [ = p(θ x r Ω)p(X r θr x )p(y r X r, θr GP, Ω) ] p({z t } t=1 Ω) (6) p({z t } t=1 Ω) = p({z t } t=1 π)p(π α)dπ = Γ(α)

r X r, θ GP r, Ω) = GP ( y r ; 0, K r + ηri ) r (10) α r X r I r r r W η r 4 K r X r θr GP 5 k r (x, x j ) = w r M ψ r,m k r,m (x, x j )

x j ) = k (p) r,m(x (p), x (p) j )k (c) r,m(x (c) r,m(x (c), x (c) j ) (1) j ) x x (p), x (c), x (p) j ) = exp ( (x(p) x (p) j ) (x (p) x

m,,..., l m,d c ) D c x (c) Θ = {θ1 x,..., θr x, θgp 1,..., θr GP} θr x = {µ r, Σ r }, θr GP = {wr, ψ r,1,.

4 Fg. 4 4 Graphcal representaton of proposed model 5 Fg. 5 Gram matrx based on multple kernel learnng ( 3 ) θr GP ( 4 ) θr x r X r ( 5 ) X r θr GP y r p({x t, y t } t=1, {z t } t=1, {θr GP } R, {θr x } R Ω) (6) R [ = p(θ x r Ω)p(X r θr x )p(y r X r, θr GP, Ω) ] p({z t } t=1 Ω) (6) p({z t } t=1 Ω) = p({z t } t=1 π)p(π α)dπ = Γ(α) Γ(α + ) R Γ( r + α/r) Γ(α/R) (7) p(θ x r Ω) = N (µ r ; m 0, Σ r /β 0 )W(Σ 1 r ; W 0, ν 0 ) (8) p(x r θr x ) = N (X r ; µ r, Σ r ) (9) p(y r X r, θ GP r, Ω) = GP ( y r ; 0, K r + ηri ) r (10) α r X r I r r r W η r 4 K r X r θr GP 5 k r (x, x j ) = w r M ψ r,m k r,m (x, x j ) (11) m=1 ω r ψ r,m [5] x, x j X r, M m=1 ψ r,m = 1 M k r,m (x, x j ) [] k r,m (x, x j ) k r,m(x (p) (p), x (p) j ) k (c), x (c) k r,m (x, x j ) = k (p) r,m(x (p), x (p) j )k (c) r,m(x (c) r,m(x (c), x (c) j ) (1) j ) x x (p), x (c), x (p) j ) = exp ( (x(p) x (p) j ) (x (p) x (p) ) j ) k r,m(x (p) (p) k r,m(x (c) (c), x (c) ) = exp j ( 1 (x(c) l (p) m ) x (c) j ) Λ(x (c) x (c) j ) Λ 1 = dag(l (c) (c) (c) m,1, l m,,..., l m,d c ) D c x (c) Θ = {θ1 x,..., θr x, θgp 1,..., θr GP} θr x = {µ r, Σ r }, θr GP = {wr, ψ r,1,..., ψ r,m, ηr} Ω = {α, m 0, W 0, β 0, ν 0, l (p) 1,..., l(p) M, l(c) 1,1,..., l(c) M,D c } x y (5) p(y {y t, x t } t=1, x, Θ, Ω) (13) = R p(y y r, X r, x, z = r, θ GP r )p(z = r x, θ x r ) p(z = r x, θ x r ) c 013 Informaton Processng Socety of Japan 4

5 p(z = r x, θ x r ) = p(x z = r, θ x r )p(z = r) p(x ) (14) p(x z = r, θ x r ) = N (x ; µ r, Σ r ), p(z = r) = 1 r p(y y r, X r, x, z = r, θr GP ) p(y y r, X r, x, z = r, θ GP r ) = N (y ; µ r,, σ r, ) (15) µ r, = k r, (K r + η ri r ) 1 y r σ r, = k r (x, x ) k r, (K r + η ri r ) 1 k r, (13) p(y {y t, x t } t=1, x, Θ, Ω) = N (y ; µ, σ ) (16) µ = R c r µ r,, σ = c r = p(z = r x, θ x r ) 3. R c rσr, MCMC-EM [8] EM z 1,..., z z t p(z t = r z \t, {x t } t=1, {y t } t=1, θ x r, θ GP r ) (17) p(y t y r,\t, X r,\t, x t, θ GP r )p(z t = r z \t, {x t } t=1, θ x r ) z \t = {z 1,..., z t 1, z t+1,..., z } y r,\t = {y : t, z = r}, X r,\t = {x : t, z = r} p(y t y r,\t, X r,\t,x t, θ GP r ) = N (y t ; µ t, σ t ) (18) µ t = k r,t[k r,\t + η ri r ] 1 y r,\t σ t = k r (x t, x t ) k r,t[k r,\t + η ri r ] 1 k r,t K r,\t X r,\t k r,t X r,\t x t p(z t = r z \t, {x t } t=1, θr x ) (19) r,\t + α/r p(x t θr x )p(θr x x \t )dθr x 1 + α = r,\t + α/r 1 + α p(xt θ x r ) :z =r, t p(x θ x r )p(θ x r )dθ x r :z =r, t p(x θ x r )p(θ x r )dθ x r r,\t X r,\t A (19) z t ψ r, w r, η r EM [5] r y r M + 1 u r,m N (u r,m ; 0, w rψ r,m K r,m ), m = 1,..., M (0) u r,m+1 N (u r,m+1 ; 0, η ri r ) (1) u r = (u r,1,..., u r,m+1 ) log p(u r ; θr GP ) = c 1 (log S r + u r S 1 r u r ) () wrψ r,1 K r,1 O... S r = wrψ r,m K r,m O ηri r = c y r, θr GP = θr GP Q Q(θ GP r, θr GP 1 ) = ( log Sr + tr ( S 1 r E[u r u r y r ; θr GP ] )) H r [I r,..., I r ] y r u r y r = H r u r E[u r u r y r ; θr GP ] E[u r u r y r ; θr GP ] = S r S r H r (H r S r H r ) 1 H r S r + S r H r (H r S r H r ) 1 y r y r (H r S r H r ) H r S r R r,1,..., R r,m+1 Q Q(θ GP r 1 w r, θr GP r ) = log(wl r ηrψ r,1... ψ r,m ) M 1 tr(k 1 ψ r,mr r,m ) 1 r,m ηr tr(r r,m+1 ) m=1 (3) ψ r,m = 1 r wr tr(k 1 r,mr r,m ) (4) M ω r = 1 r M m=1 1 ψ r,m tr(k 1 r,mr r,m ) (5) η r = 1 r tr(r r,m+1 ) (6) M m=1 ψ r,m = 1 c 013 Informaton Processng Socety of Japan 5

6 6 6:4 Fg. 6 ranng and test segments for evaluaton ψ r,m (3) (4) (5) (6) Σ r Σ r p(σ 1 r X r, {z t : z t = r}) = W(Σ 1 r ; W r, ν r ) (7) W 1 r β r = β 0 + r, ν r = ν 0 + r x r = 1 r r =1 = W 1 µ r x r,, (x r, X r ) r 0 + =1 x r, x r, + β 0 r ( x r µ β 0 )( x r µ 0 ) r µ r p(µ r X r,{z t : z t = r}) = N (µ r ; m r, Σ r /β r ) (8) 4. m r = 1 β r (β 0 m 0 + r x r ) 1 J-pop 10 4 MIDI khz 16 (1) (sec.) (MIDI ) (sec.) 3 x t x (p) t,1 =, x(c) t,1 =, x (c) t, = () 6 6:4 7 Fg. 7 ±σ State assgnments n tranng segments and predctve dstrbutons n test segments z 1,..., z k-means R θ x r M = 30 w r = 100, ψ r,1 = 1/M,..., ψ r,m = 1/M, η r = 10 (r = 1,..., R) η r α = 1, β 0 = 0.1, ν 0 = D + 1 D m 0 W 0 ν 0 l (p) 1:10 = l(p) 11:0 = l(p) 1: l (c) 1:10,1 = 1, l(c) 11:0,1 =, l(c) 1:30,1 = 3, l(c) 1:10, = 0.1, l (c) 11:0, = 0., l(c) 1:30, = ψ r, wr, ηr EM R (16) c 013 Informaton Processng Socety of Japan 6

7 8 Fg. 8 Gaussan dstrbutons traned n nput varable space µ ±σ HMM R 8 R = 10 R = 50 (14) R = 50 R r r (7) R 9 Fg. 9 Root mean square errors between volume contours and predctve means (16) (RMSE) 9 RMSE R = 50 9 RMSE RMSE x (c) t,3 =, x(c) t,4 =, x (c) t,5 =, x(c) t,6 = 5. c 013 Informaton Processng Socety of Japan 7

8 HMM ψ r,m F 0 A (19) p(xt θr x ) :z log p(x =r, t θr x )p(θr x )dθr x :z p(x =r, t θr x )p(θr x )dθr x (9) c = log B(W r,t, ν r,t ) B(W r,\t, ν r,\t ) D log β 0 + r,\t + 1 β 0 + r,\t B(W, ν) B(W, ν) W ν ( νd π D(D 1) 4 D =1 Γ ( ν + 1 ) ) 1 (9) β r,t = β 0 + r,\t + 1, β r,\t = β 0 + r,\t ν r,t = ν 0 + r,\t + 1, ν r,\t = ν 0 + r,\t 1 ( r,\t ) x r,t = x t + x r,, x r,\t = 1 r,\t r,\t + 1 r,\t =1 =1 r,\t W 1 r,t = W x r, x r, + x t x t W 1 r,\t = W r,\t x r, x r, =1 + β 0( r,\t + 1) ( x r,t µ β 0 )( x r,t µ 0 ) r,t =1 + β 0 r,\t ( x r,\t µ β 0 )( x r,\t µ 0 ) r,\t x r, x r, X r,\t D [1] SRAIGH Vol. 43, No., pp (00). [] Nakano,. et al.: An Automatc Sngng Skll Evaluaton Method for Unknown Melodes Usng Ptch Interval Accuracy and Vbrato Features, n Proc. ICSLP 006, pp (006). [3] Mayor, O. et al.: he Sngng utor: Expresson Categorzaton and Segmentaton of the Sngng Voce, n Proc. AES 11st Conventon (006). [4] VocaLstener Vol. 5, No. 1, pp (011). [5] Bonada, J. et al.: Synthess of the Sngng Voce by Performance Samplng and Spectral Models, IEEE Sgnal Processng Magazne, Vol. 4, pp (007). [6] Kako,. et al.: Automatc Identfcaton for Sngng Style based on Sung Melodc Contour Characterzed n Phase Plane, n Proc. ISMIR 009, pp (009). [7] Fukayama, S. et al.: Orpheus: Automatc Composton System Consderng Prosody of Japanese Lyrcs, n Proc. ICEC 009, pp (009). [8] Nakano,. et al.: VocaLstener: A Sngng Synthess System Able to Mmc a User s Sngng n terms of Voce mbre Chages as well as Ptch and Dynamcs, n Proc. ICASSP 011, pp (011). [9] Mase, A. et al.: HMM-based Sngng Voce Synthess System Usng Ptch-shfted Pseudo ranng Data, n Proc. INERSPEECH 010, pp (010). [10] Sundberg, J.: he Scence of the Sngng Voce, Northern Illnos Unversty Press, Illnos (1987). [11] Proctor, D. F.: Breathng, Speech and Song, Sprnger- Verlag, New York (1980). [1] Bouhuys, A. et al.: Knetc Aspects of Sngng, Journal of Appled Physology, Vol. 1, pp (1966). [13] Sundberg, J.: Actvaton of the Daphragm n Sngng, n Proc. SMAC 1983, pp (1983). [14] su, W. H. et al.: Method and System on Detectng Abdomnals for Sngng, n Proc. IEEE EMBC 013 (013). [15] Rubn, H. J. et al.: Vocal Intensty, Subglottc Pressure and Ar Flow Relatonshps n Sngers, Fola Phonatr, Vol. 19, No. 6, pp (1967). [16] Cleveland,. et al.: Acoustc Analyss of hree Male Voces of Dfferent Qualty, n Proc. SMAC 1983, pp (1983). [17] Scully, C. et al.: Smulaton of Sngng wth a Composte Model of Speech Producton, n Proc. SMAC 1983, pp (1983). [18] eramura, K. et al.: Gaussan Process Regresson for Renderng Musc Performance, n Proc. ICMPC 008 (008). [19] 75, 3R-7 pp (013). [0] Kameoka, H. et al.: A Statstcal Model of Speech F0 Contours, n Proc. SAPA 010, pp (010). [1] Ohsh, Y. et al.: A Stochastc Model of Sngng Voce F0 Contours for Characterzng Expressve Dynamc Components, n Proc. INERSPEECH 01 (01). [] Koryama,. et al.: Frame-level Acoustc Modelng based on Gaussan Process Regresson for Statstcal Nonparametrc Speech Synthess, n Proc. ICASSP 013, pp (013). [3] Meeds, E. et al.: An Alternatve Infnte Mxture of Gaussan Process Experts, n Proc. NIPS 006 (006). [4] Plkngton, N. C. V. et al.: Gaussan Process Experts for Voce Converson, n Proc. INERSPEECH 011, pp (011). [5], -Q-4 pp (010). [6] Henter, G. E. et al.: Gaussan Process Dynamcal Models for Nonparametrc Speech Representaton and Synthess, n Proc. ICASSP 01, pp (01). [7] Rasmussen, C. E. et al.: Infnte Mxtures of Gaussan Process Experts, n Proc. NIPS 00 (00). [8] Andreu, C. et al.: An Introducton to MCMC for Machne Learnng, Machne Learnng, Vol. 50, No. 1-, pp (003). c 013 Informaton Processng Socety of Japan 8

IPSJ SIG Techncal Report 歌声データベース歌声の波形スペクトル抽出基本周波数抽出 HMM メルケプストラムラベル HMM の学習対数基本周波数 c 学習部コンテキスト依存モデル c ( 合成部楽譜ラベル変換ラベル... メルケプストラムパラメータ生成 ML

IPSJ SIG Techncal Report 歌声データベース歌声の波形スペクトル抽出基本周波数抽出 HMM メルケプストラムラベル HMM の学習対数基本周波数 c 学習部コンテキスト依存モデル c ( 合成部楽譜ラベル変換ラベル... メルケプストラムパラメータ生成 ML IPSJ SIG Techncal Report HMM HMM HMM HMM 60 Vbrato Modelng for HMM-based Sngng Voce Synthess Tomohko Yamada, Satoru Muto, Yoshhko ankaku, Shnj Sako and Kech Tokuda HMM-based sngng voce synthess can automatcally