676 Vol. 31 No. 7, pp , Incremental Noise Estimation in Outdoor Auditory Scene Analysis using a Quadrocopter with a Microphone A

Size: px

Start display at page:

Download "676 Vol. 31 No. 7, pp , Incremental Noise Estimation in Outdoor Auditory Scene Analysis using a Quadrocopter with a Microphone A"

あけなおつちかね
4 years ago
Views:

1 676 Vol. 31 No. 7, pp , Incremental Noise Estimation in Outdoor Auditory Scene Analysis using a Quadrocopter with a Microphone Array Keita Okutani 1, Takami Yoshida 1, Keisuke Nakamura 2 and Kazuhiro Nakadai 1 2 This paper addresses sound source localization using an aerial vehicle with a microphone array in an outdoor environment to realize outdoor auditory scene analysis. It, for instance, aims at finding distressed people in a disaster situation. In such an environment, noise is quite loud and dynamically-changing, and conventional microphone array techniques studied in the field of indoor robot audition are of less use. We, thus, proposed MUltiple SIgnal Classification based on incremental Generalized EigenValue Decomposition (igevd-music). It can deal with dynamically-changing high power noise by introducing incrementally-estimated noise correlation matrices. We developed a prototype system for the outdoor auditory scene analysis based on the proposed method using the Parrot AR.Drone with an 8 ch microphone array and a Kinect device. Experimental results using the prototype system showed that dynamically-changing noise is properly suppressed with the proposed method even when the signal-to-noise ratio is less than 0 db in an outdoor/indoor environment with the hovering/moving AR.Drone. Key Words: Sound Source Localization, Outdoor, Quadrocopter 1. Computational Auditory Scene Analysis, CASA Robot Audition [1] [2] [3] Graduate School of Information Science and Engineering, Tokyo Institute of Technology 2 Honda Research Institute Japan Co., Ltd. J-STAGE Fig. 1 Auditory scene analysis in an outdoor environment Fig. 1 3 [4] [5] JRSJ Vol. 31 No Sept., 2013

2 677 Acoustic Vector Sensor AVS Unmanned Aerial Vehicle UAV [6] [7] [8] [9] [10] Root-MUSIC [11], ES- PRIT [12] [13] Multiple Signal Classification MUSIC [14] Signal-to-Noise Ratio SNR MUSIC MUSIC Generalized EigenValue Decomposition, GEVD MUSIC GEVD-MUSIC 10 [db] [15] [16] GEVD-MUSIC GEVD-MUSIC incremental GEVD-MUSIC igevd-music [17] 24 [bit] A/D Kinect 2 igevd-music igevd-music 2. 1 Standard EigenValue Decomposition, SEVD MUSIC SEVD-MUSIC SEVD-MUSIC GEVD-MUSIC GEVD-MUSIC igevd-music Table igevd-music M f X M X(ω, f) = N(ω, f) = LX A l (ω, θ l )S l (ω, θ l, f) + N(ω, f) 1 l=1 JX A n j (ω, θ j )N j (ω, θ j, f) + N d (ω, f) 2 j=1 A S l l ω L θ l l N θ j N j J N d A n SEVD-MUSIC S l N j S l L + J = M N j S l Table 1 Variables and parameters M number of microphones (8) L number of sound sources (L < M) m microphone index (1 m M) l sound source index (1 l L) f frame index ω frequency bin index θ l l-th sound source direction ψ direction of a steering vector X m (ω, f) observation for m-th microphone at f-th frame X(ω, f) [X 1 (ω, f), X 2 (ω, f),..., X M (ω, f)] T G(ω, ψ) steering vector R(ω, f) correlation matrix of X(ω, f) ( C M M ) e m(ω, f) m-th Eigen vector of R(ω, f) E(ω, f) [e 1 (ω, f),..., e M (ω, f)] λ m (ω, f) m-th Eigenvalue of R(ω, f) (λ 1 λ 2... λ M ) Λ(ω, f) diag(λ 1,...λ M ) ω l ω h minand max frequencies for MUSIC (500, 2,800 [Hz]) T R number of frames for averaging correlation matrices

3 678 Table 2 Configuration of moving microphone array hardware device size & weight quadrocopter AR.Drone [mm], payload:130 [g] recording device RASP [mm], 104 [g] microphone MEMS [mm], 3 [g] Fig. 2 Noise correlation matrix estimation in igevd-music GEVD-MUSIC igevd-music N L < M G(ω, ψ) ψ ψ 5 R(ω, f) T R R(ω, f) = 1 fx X(ω, τ)x (ω, τ) T R τ=f T R 3 K GEVD-MUSIC K igevd-music Fig. 2 K(ω, f) = 1 T N f f X s τ=f f s T N X(ω, τ)x (ω, τ) 4 T N K f s R K T N f s R R 4 igevd-music R(ω, f) K 1 (ω, f) GEVD R(ω, f) K SEVD R(ω, f) igevd GEVD SEVD K 1 (ω, f)r(ω, f) = E(ω, f)λ(ω, f)e (ω, f) 5 K 1 (ω)r(ω, f) = E(ω, f)λ(ω, f)e (ω, f) R(ω, f) = E(ω, f)λ(ω, f)e (ω, f) 6 7 Λ(ω, f) E(ω, f) Λ(ω, f) e m (ω, f) ψ G(ω, ψ) MUSIC P (ω, ψ, f) P (ω, ψ, f) = G (ω, ψ)g(ω, ψ) P M m=l+1 G (ω, ψ)e m(ω, f) 8 L P (ω, ψ, f) ω P (ψ, f) = 1 ω H ω L + 1 ω H X ω=ω L P (ω, ψ, f) 9 ω H ω L P (ψ, f) L ψ P (ψ, f) 3. Fig. 3 PC 3. 1 Fig. 4 Table 2 AR.Drone 130 [g] AR.Drone 115 [g] Kinect 35 [mm] ch PC Fig. 3 PC ROS Kinect igevd-music HARK [18] JRSJ Vol. 31 No Sept., 2013

679 Fig. 3 System structure for outdoor auditory scene analysis Fig.

5 Position of a vehicle and spekaers igevd-music SEVD-MUSIC, GEVD-MUSIC AR.Drone Kinect AR.Drone 4. 1 4. igevd-music Ex1. Ex2.

php?contentid=7& categoryid=4&page name=rasp-24 a) Indoor b) Outdoor Table 3 Fig.

4 679 Fig. 3 System structure for outdoor auditory scene analysis Fig. 4 Quadrocpoter with microphone array a) Hovering b) Moving Fig. 5 Position of a vehicle and spekaers igevd-music SEVD-MUSIC, GEVD-MUSIC AR.Drone Kinect AR.Drone igevd-music Ex1. Ex2. Ex3. + Fig [m] Fig. 5a) 12 Fig. 5b) categoryid=4&page name=rasp-24 a) Indoor b) Outdoor Table 3 Fig. 6 Experimental environments Loudness of sound and noise sources sound source indoor outdoor propeller utterance sound level[db] [m] Fig. 6 Table SEVD-MUSIC GEVD-MUSIC igevd-music P (ψ, f) MUSIC 2 3 P (ψ, f) 4 igevd-music T N f s

5 680 Fig. 7 Spatial spectrograms and localization results (Indoor-hovering) Fig. 8 Spatial spectrograms and localization results (Indoor-moving) Fig. 9 Spatial spectrograms and localization results (Outdoor-hovering) MUSIC P (ψ, f) Fig. 7 9 b) d) MUSIC Kinect Fig. 7 9 a) 2 Kinect Kinect 3.5 [m] 1.3 [cm] Kinect Kinect AR.Drone Table 4 Noise suppression performance Method std. dev. SEVD-MUSIC 0.14 GEVD-MUSIC 0.12 igevd-music Table 5 Utterance-based localization performance (%) Condition SEVD GEVD igevd LAR LCR LAR LCR LAR LCR Indoor-Hovering Indoor-Moving Outdoor-Hovering Fig. 7 9 Fig. 7 AR.Drone 89 [db] SEVD GEVD igevd Fig. 8 AR.Drone GEVD 160 igevd GEVD Fig. 9 JRSJ Vol. 31 No Sept., 2013

6 681 Fig. 10 Normalized histograms Fig. 11 Difference between speaking area and non-speaking area 45 [db] [db] SEVD GEVD igevd LAR Localization Accuracy Rate LCR Localization Correct Rate LAR = N t N S N D N I N t LCR = N t N S N D N t N t N S N D N I LAR LCR ±0.2 ±5 ±5 Table 5 igevd LAR LCR igevd P (ψ, f) Fig. 10 Table 4 Fig P P Table 4 c) igevd igevd-music igevd-music Fig. 2 T N f s igevd- MUSIC P (ψ, f) T N f s Fig. 11 T N f s P T N f s [s] igevd-music

682 (a) Spatial spectrograms, localization

3 2 1 1 1 2 [m] 2 24 igevd- MUSIC T N = 90

7 682 (a) Spatial spectrograms, localization results and pictures Fig. 12 (b) Reference Outdoor localization experiment T N [m] 2 24 igevd- MUSIC T N = 90 [frame] f s = 140 [frame] Fig. 12 Fig. 12 (a) MUSIC Fig. 12 (b) LAR = 21% LCR = 42% N D [19] [20] 5. igevd-music Kinect igevd-music 15 30% 80% MUSIC JRSJ Vol. 31 No Sept., 2013

683 24118702 22700165 [ 1 ] I. Hara, F. Asano, H. Asoh, J. Ogata, N. Ichimura, Y. Kawai, F. Kanehiro, H. Hirukawa and K.

2404 2410, 2004. [ 2 ] J.-M. Valin, F. Michaud and J.

[ 3 ] Simon THOMPSON 64ch 32 AI Challenge pp.3 8, 2010. [ 4 ] 12 pp.2381 2384, 2011. [ 5 ] K. Okutani, T. Yoshida, K. Nakamura and K.

2012), pp.3288 3293, 2012. [ 6 ] B. Kaushik, D. Nance and K. K. Ahuj: A review of the role of acoustic sensors in the modern battlefield, 11th AIAA/CEAS Aeroacoustics Conference (26th AIAA Aeroacoustics Conference), pp.

8 [ 1 ] I. Hara, F. Asano, H. Asoh, J. Ogata, N. Ichimura, Y. Kawai, F. Kanehiro, H. Hirukawa and K. Yamamoto: Robust speech interface based on audio and video information fusion for humanoid HRP-2, Proc. of IEEE/RAS International Conference on Intelligent Robots and Systems (IROS 2004), pp , [ 2 ] J.-M. Valin, F. Michaud and J. Rouat: Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering, Robotics and Autonomous Systems Journal, vol.55, no.3, pp , [ 3 ] Simon THOMPSON 64ch 32 AI Challenge pp.3 8, [ 4 ] 12 pp , [ 5 ] K. Okutani, T. Yoshida, K. Nakamura and K. Nakadai: Outdoor auditory scene analysis using a moving microphone array embedded in a quadrocopter, Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2012), pp , [ 6 ] B. Kaushik, D. Nance and K. K. Ahuj: A review of the role of acoustic sensors in the modern battlefield, 11th AIAA/CEAS Aeroacoustics Conference (26th AIAA Aeroacoustics Conference), pp.1 13, [ 7 ] Hans-Elias de Bree: Acoustic vector sensors increasing UAV s situational awareness, SAE Technical Paper, pp , [ 8 ] B. D. Van Veen and K. M. Buckley: Beamforming: a versatile approach to spatial filtering, IEEE ASSP Magazine, vol.5, no.2, pp.4 24, [ 9 ] J. Burg: Maximum entropy spectral analysis, 37th Meeting Society Exploration Geophysicists, 1967; reprinted in Modern Spectrum Analysis (D.G. Childers, ed.). pp.34 39, IEEE press, [10] S.S. Reddi: Multiple source location a digital approach, IEEE Trans. on Aerospace and Electronic Systems, AES-15:95 105, [11] B.D. Rao and K.V.S. Hari: Performance analysis of rootmusic, IEEE Trans. on Acoustics, Speech, and Signal Processing, vol.37, no.12, pp , [12] R. Roy and T. Kailath: Esprit estimation of signal parameters via rotational invariance techniques, IEEE Trans. on Acoustics, Speech, and Signal Processing, vol.37, no.7, pp , [13] A. Swami and J. M. Mendel: Cumulant-based approach to harmonic retrieval and related problems, IEEE Trans. on Signal Processing, vol.39, no.5, pp , [14] R. Schmidt: Multiple emitter location and signal parameter estimation, IEEE Trans. on Antennas and Propagation, vol.34, no.3, pp , [15] K. Nakamura, K. Nakadai, F. Asano, Y. Hasegawa and H. Tsujino: Intelligent sound source localization for dynamic environments, Proc. of IEEE/RSJ Int l Conf. on Intelligent Robots and Systems (IROS 2009), pp , [16] K. Nakamura, K. Nakadai, F. Asano and G. Ince: Intelligent sound source localization and its application to multimodal human tracking, Proceedings of 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2011), pp , [17] 30 DVD-ROM 3D1-2, [18] K. Nakadai, T. Takahashi, H.G. Okuno, H. Nakajima, Y. Hasegawa and H. Tsujino: Design and implementation of robot audition system HARK open source software for listening to three simultaneous speakers, Advanced Robotics, vol.24, no.5 6, pp , [19] G. Ince, K. Nakadai and K. Nakamura: Online learning for template-based multi-channel ego noise estimation, Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2012), pp , [20] T. Yoshida and K. Nakadai: Active audio-visual integration for voice activity detection based on a causal bayesian network, Proceedings of the 2012 IEEE RAS International Conference on Humanoid Robots (Humanoids 2012), pp , Keita Okutani Keisuke Nakamura 2007 University of Strathclyde Takami Yoshida Kazuhiro Nakadai NTT NTT JST ERATO IEEE

IPSJ SIG Technical Report Vol.2015-MUS-107 No /5/23 HARK-Binaural Raspberry Pi 2 1,a) ( ) HARK 2 HARK-Binaural A/D Raspberry Pi 2 1.

IPSJ SIG Technical Report Vol.2015-MUS-107 No /5/23 HARK-Binaural Raspberry Pi 2 1,a) ( ) HARK 2 HARK-Binaural A/D Raspberry Pi 2 1. HARK-Binaural Raspberry Pi 2 1,a) 1 1 1 2 3 () HARK 2 HARK-Binaural A/D Raspberry Pi 2 1. [1,2] [2 5] () HARK (Honda Research Institute Japan audition for robots with Kyoto University) *1 GUI ( 1) Python