27 5) STRAIGHT ) STRAIGHT 8) 3 STRAIGHT ),6),2) 7) 7),9) 5) STRAIGHT 5),7) 2.. spline ) ms ) STRAIGHT (db) ERB N(Effective Rectangul

Size: px

Start display at page:

Download "27 5) STRAIGHT ) STRAIGHT 8) 3 STRAIGHT ),6),2) 7) 7),9) 5) 2. 2. STRAIGHT 5),7) 2.. spline 2..2 6) ms 2..3 4) STRAIGHT (db) ERB N(Effective Rectangul"

りえかんざとばる
7 years ago
Views:

1 STRAIGHT Perceptual study on design reuse of voice identity and singing style based on singing voice morphing HIDEKI KAWAHARA, TAICHI IKOMA, MASANORI MORISE, TORU TAKAHASHI, KEN ICHI TOYODA 4 and HARUHIRO KATAYOSE 3 Investigations on singing voice morphing has been conducted to establish a design reuse framework based on a high-quality speech analysis, modification and resynthesis system STRAIGHT. STRAIGHT enables high-quality speech reconstruction using only three parameters; fundamental frequency, time-frequency representation, and time-frequency aperiodicity map. In this paper, an extension of STRAIGHT-based morphing, which enables individual control of morphing rate using five parameters (in addition to three parameters mentioned above, time and frequency axes were introduced), was implemented to test design reuse of singers voice identity and their singing style. Subjective evaluations of two sets of manipulated samples were conducted. First set was generated from a male student and a professional male singer and the second set was generated from two professional female singers. Test results illustrated that intended reuse was perceptually verified and generated samples were generally in high-quality.. Faculty of Systems Engineering, Wakayama University 2 Graduate School of Systems Engineering, Wakayama University 3 School of Science and Technology, Kwansei Gakuin Univ. 4 Graduate School of Science and Technology, Kwansei Gakuin Univ. STRAIGHT 7) 8),6),2) STRAIGHT 3) STRAIGHT channel VOCODER 4)

resynthesis system STRAIGHT. STRAIGHT enables high-quality speech reconstruction using only three parameters; fundamental frequency, time-frequency representation, and time-frequency aperiodicity map.

2 27 5) STRAIGHT ) STRAIGHT 8) 3 STRAIGHT ),6),2) 7) 7),9) 5) STRAIGHT 5),7) 2.. spline ) ms ) STRAIGHT (db) ERB N(Effective Rectangular Bandwidth) 2). ),2) 9) 2..4 Matlab STRAIGHT Matlab 2.2 STRAIGHT

.3 4) STRAIGHT (db) ERB N(Effective Rectangular

2.2. - - - STRAIGHT 8) - - 2.2.2 5 2 r λ 5 (r (λ), r 2 (λ), r 3 (λ), r 4 (λ), r 5 (λ)) 2 r k () =, r k () = k {, 2, 3, 4, 5} λ λ 2 r k (λ ) r k (λ 2 ) Matlab 5 Fig.

3 STRAIGHT 8) r λ 5 (r (λ), r 2 (λ), r 3 (λ), r 4 (λ), r 5 (λ)) 2 r k () =, r k () = k {, 2, 3, 4, 5} λ λ 2 r k (λ ) r k (λ 2 ) Matlab 5 Fig. GUI tool for editing time-frequency markers ) 5) 5) ) σ 5 ms 2 ms 65 Hz 25 Hz σ STRAIGHT GUI

4 ) STRAIGHT 4) 2) r timbre r way 4. 2 Fig. 2 GUI for subjective evaluation of morphed singing 4. RWC 8) So Long RWC-MDB-P-2 No.64 A 2 B B audio-technica ATH-A SHURE SM57-LCE 44.kHz 6bit A B 2 r timbre r way A B 66 A B 7 Hz audio-technica ATH-A A B

5 A B 2 GUI r timbre 25% 75% r way 3 6) Evaluated Value of σ timbre Timbre of Singing Voice timbre 3 Fig. 3 Timber rating results for r timbre manipulation Way of Singing.2 Evaluated Value of σ way way 4 Fig. 4 Style rating results for r way manipulation Naturality of Singing Voice 5 Evaluated Value of Naturality timbre 5 Fig. 5 Naturalness rating for r timbre manipulation

2 Evaluated Value of σ way.8.6.4.2.2.2.2.4.6.8.2 way 4 Fig.

6 27 Evaluated Value of Naturality Naturality of Singing Voice way 6 Fig. 6 Naturalness rating for r way manipulation r way r timbre A B 3 A B (Love affair) σ timbre A or B way 7 A B Fig. 7 Singer identification map for r timber and r way manipulations. Dark color represents singer A and white color represents singer B Fig. 8 Evaluated Value of r timbre Timbre of Singing Voice r timbre Timber rating results for r timbre manipulation % 9% 8

7 Singer identification map for r timber and r way manipulations. Dark color represents singer A and white color represents singer B Fig.

7 Way of Singing 5 Naturality of Singing Voice Evaluated Value of r way r way 9 Fig. 9 Style rating results for r way manipulation Evaluated Value of Naturality r way.2 Fig. Naturalness rating for r way manipulation 5 Naturality of Singing Voice Evaluated Value of Naturality r timbre Fig. Naturalness rating for r timbre manipulation A B Fig. 2 Singer identification map for r timber and r way manipulations. Dark color represents singer A and white color represents singer B

Naturalness rating for r way manipulation 5 Naturality of Singing Voice Evaluated Value of Naturality 4 3 2.2.2.4.6.8.2 r timbre Fig.

8 27 5. STRAIGHT ) Abe, T., Kobayashi, T. and Imai, S.: Harmonics estimation based on instantaneous frequency and its application to pitch determination, IEICE Trans. Information and Systems, Vol.E78-D, No.9, pp (995). 2) Abe, T., Kobayashi, T. and Imai, S.: The IF Spectrogram: A New Spectral Representation, Proc. ASVA-97, Tokyo, pp (997). 3) Bregman, A.S.: Auditory Scene Analysis, MIT Press, Cambridge, MA (99). 4) Dudley, H.: Remaking speech, J. Acoust. Soc. Am., Vol., No.2, pp (939). 5) Kawahara, H.: STRAIGHT, Exploration of the other aspect of VOCODER: Perceptually isomorphic decomposition of speech sounds, Acoust. Sci. & Tech., Vol.27, No.6, pp (26). 6) Kawahara, H., de Cheveigné, A., Banno, H., Takahashi, T. and Irino, T.: Nearly Defectfree F Trajectory Extraction for Expressive Speech Modifications based on STRAIGHT, Interspeech 5, Lisboa, pp (25). 7) Kawahara, H., Masuda-Katsuse, I. and de Cheveigné, A.: Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequencybased F extraction, Speech Communication, Vol.27, No.3-4, pp (999). 8) Kawahara, H. and Matsui, H.: Auditory morphing based on an elastic perceptual distance metric in an interference-free time-frequency representation, Proc. ICASSP 23, Vol. I, Hong Kong, pp (23). 9) Kawahara, H., Katayose, H., de Cheveigné, A. and Patterson, R.D.: Fixed Point Analysis of Frequency to Instantaneous Frequency Mapping for Accurate Estimation of F and Periodicity, Proc. Eurospeech 99, Vol.6, pp (999). ) Matsui, H. and Kawahara, H.: Investigation of Emotionally Morphed Speech Perception and its Structure using a High Quality Speech Manipulation System, Proc. Eurospeech 3, Geneva, pp (23). ) Mayer, H. and Steger, C.: A New Approach For Line Extraction and its Integration in a Multi-Scale, Multi-Abstraction-Level Road Extraction System, Mapping Buildings, Roads and other Man-Made Structures from Images (Leberl, F., Kalliany, R. and Gruber, M., eds.), Wien, R. Oldenbourg Verlag, pp (996). 2) Moore, B. C.J.: An introduction to the psychology of hearing: fifth edition, Academic Press (23). 3) Smith, D. R., Patterson, R. D., Turner, R., Kawahara, H. and Irino, T.: The processing and perception of size information in speech sounds, J. Acoust. Soc. Am., Vol. 7, No., pp (25). 4) Titze, I. R.: Principles of voice production, Prentice Hall (994). 5) STRAIGHT Vol.26-MUS-64 (26). 6) Yonezawa, T., Suzuki, N., Mase, K. and Kogure, K.: HandySinger: Expressive Singing Voice Morphing using Personified Handpuppet Interface, Proc. NIME25, Hamamatsu, pp.2 26 (25). 7) STRAIGHT Vol.43, No.2, pp (22). 8) RWC : Vol.45, No.3, pp (24). 9) - CrestMuse - pp.d 4 (26). 2) H-23-4, pp (25).

4) Dudley, H.: Remaking speech, J. Acoust. Soc. Am., Vol., No.2, pp.69 77 (939). 5) Kawahara, H.

IPSJ SIG Technical Report Vol.2012-MUS-94 No.3 Vol.2012-SLP-90 No /2/ DTM 200 GUIN-Resonator: A system synthesizing voice with the styl

IPSJ SIG Technical Report Vol.2012-MUS-94 No.3 Vol.2012-SLP-90 No /2/ DTM 200 GUIN-Resonator: A system synthesizing voice with the styl 1 1 2 1 DTM 200 GUIN-Resonator: A system synthesizing voice with the style of Amami folk songs Daisuke Suguru, 1 Takashi Baba, 1 Masanori Morise 2 and Haruhiro Katayose 1 The recent spread of Karaoke and