IPSJ SIG Technical Report Vol.2012-MUS-96 No /8/10 MIDI Modeling Performance Indeterminacies for Polyphonic Midi Score Following and

Size: px

Start display at page:

Download "IPSJ SIG Technical Report Vol.2012-MUS-96 No /8/10 MIDI Modeling Performance Indeterminacies for Polyphonic Midi Score Following and"

みりあしまむね
6 years ago
Views:

1 MIDI Modeling Performance Indeterminacies for Polyphonic Midi Score Following and Its Application to Automatic Accompaniment Nakamura Eita 1 Yamamoto Ryuichi 2 Saito Yasuyuki 3 Sako Shinji 2 Sagayama Shigeki 1 Abstract: Score following plays an important role in automatic accompaniment, which is an automated performance of accompaniment in synchrony with human performances. This paper describes the score following capable of following performances with ornaments and improvised phrases. We construct a probabilistic model of ornaments based on hidden markov model and discuss a method of describing the structure of more indeterminate improvisational phrases. A score following algorithm based on the model is proposed and its effectiveness is evaluated using human-played performances. An automatic accompaniment system using the algorithm is built and its operation is tested. Keywords: score following, automatic accompaniment, performance indeterminacies, hidden markov model, tempo estimation The University of Tokyo, Hongo, Bunkyo-ku, Tokyo , Japan 2 Nagoya Institute of Technology, Gokiso-cho, Showa-ku, Nagoya , Japan 3 KisarazuNCT, Kiyomidai-Higashi, Kisarazu, Chiba , Japan Dannenberg [1] Vercoe [2] [3] c 2012 Information Processing Society of Japan 1

2 Fig. 1 1 Various realizations of a trill. [4], [5] MIDI Raphael [6] Cont [4] MIDI [5] MIDI MIDI [8] a) b) c) d) e) f) a) d) [3], [5], [7], [8] e) f) 1 f) 2.2 τ t (onset time) X = {(τ i, c i )} I i=1 I i τ i i c i c i c i c i c i S = {(t m, s m )} M m=1 m t m c 2012 Information Processing Society of Japan 2

3 s m MIDI s m MIDI s m [8] M X S X S i M i M M τ im 3. HMM HMM 2 p(t m, s m ) p(t m, s m i m, i m 1, t m 1 ) (δt m, s m ) = p(t m, s m i m, i m 1, t m 1 ) (1) δt m = t m t m 1 (inter-onset interval, ioi) p(i m, t m i m 1, t m 1 ) δt m S = {(t m, s m )} M m=1 p(s) = Q = p(s Q)p(Q) (2) i 1,,i m m=1 M a im 1,i m (δt m, s m ) (3) Q = {i m } M m=1 a im 1,i m = p(i m, t m i m 1, t m 1 ) a im 1,i m (δt m, s m ) Hidden Markov Model, HMM HMM ioi (δt m, s m ) = b (ioi) i m 1,i m (δt m )b (evt) i m (s m ) (4) 3.3 a im 1,i m a im 1,i m = δ im 1+1,i m 0 a im 1,i m i m = i m i m = i m 1 i m < i m 1 i m = i m 1 + d d 1 a im 1,i m HMM 2 ioi c 2012 Information Processing Society of Japan 3

4 2 Fig. 2 HMM i i Topology of state transition probability for the performance HMM. The i th state corresponds to the i th musical action (chord, ornament etc.) of the performace score. 4. HMM 4.1 i HMM ioi ioi 35 msec [9] b (ioi) i,i (δt) δt 35 msec b (evt) i (s) s HMM 1 *1 shake 3 ioi 30 < δt < 200 msec b (ioi) i,i (δt) b (evt) i (s) s * ioi 1 HMM 1 ( ) 1 HMM ioi 30 < δt < 100 msec b (ioi) i,i (δt) HMM HMM HMM HMM ioi HMM HMM 5.2 c 2012 Information Processing Society of Japan 4

5 HMM HMM ( ) HMM HMM 5.3 HMM ioi HMM X S Q = {i m } M m=1 Bayes argmax Q = argmax i 1,,i M p(q S) = argmax [p(s Q)p(Q)] (5) Q [ M ] a im 1,i m (δt m, s m ) m=1 (6) HMM Viterbi t M 1 i M 1 ˆp im 1 = max i 1,,i M 2 ˆp im [ M 1 m=1 a im 1,i m (δt m, s m ) ] (7) = max i M 1 [ˆpiM 1 a im 1,i M b im 1,i M (δt M, s M ) ] (8) Viterbi [8] HMM 6.2 HMM r im = (τ im+1 τ im )/(t im+1 t im ) HMM [8] MIDI 3 Beethoven 3 4 Mozart 2 c 2012 Information Processing Society of Japan 5

6 8. 3 Fig. 3 Estimation of score location for a performance with trills. In the piano roll, the vertical lines show onsets, and the blue bold lines show onsets where the score location estimation is updated. HMM HMM MusicXML midi HMM 4 Fig. 4 Estimation of score location for a performance with arpeggi. Beethoven 7.2 Eurydice [8], [10] [1] R. Dannenberg, An on-line algorithm for real-time accompaniment, Proc. ICMC, pp , [2] B. Vercoe, The synthetic performer in the context of live performance, Proc. ICMC, pp , [3] N. Orio et al., Score Following: State of the Art and New Developments, in New Interfaces for Musical Expression, [4] A. Cont, ANTESCOFO: Anticipatory synchronization and control of interactive parameters in computer music, Proc. ICMC, [5] D. Schwarz, Nicola Orio and N. Schnell, Robust Polyphonic Midi Score Following with Hidden Markov Models, Proc. ICMC, [6] C. Raphael, Music Plus One: A system for expressive and flexible musical accompaniment, Proc. ICMC, [7], HMM MIDI,, MUS, pp , [8] Eurydice:, [9] MIDI,, 48(1), pp , [10] Eurydice:, 96, c 2012 Information Processing Society of Japan 6

( ) [1] [4] ( ) 2. [5] [6] Piano Tutor[7] [1], [2], [8], [9] Radiobaton[10] Two Finger Piano[11] Coloring-in Piano[12] ism[13] MIDI MIDI 1 Fig. 1 Syst

( ) [1] [4] ( ) 2. [5] [6] Piano Tutor[7] [1], [2], [8], [9] Radiobaton[10] Two Finger Piano[11] Coloring-in Piano[12] ism[13] MIDI MIDI 1 Fig. 1 Syst 情報処理学会インタラクション 2015 IPSJ Interaction 2015 15INT014 2015/3/7 1,a) 1,b) 1,c) Design and Implementation of a Piano Learning Support System Considering Motivation Fukuya Yuto 1,a) Takegawa Yoshinari 1,b) Yanagi