IPSJ-CVIM

Size: px

Start display at page:

Download "IPSJ-CVIM"

くにひとわにべ
5 years ago
Views:

1 00 Periodic Image Sequence Reconstruction from Low Frame-rate Videos for Gait Recognition Naoki Akae, Al Mansur, Yasushi Makihara and Yasushi Yagi This paper describes a method of periodic temporal super resolution and its application to low frame-rate gait recognition. The existing reconstructionbased methods of periodic temporal super resolution often suffer from two problems: wagon-wheel effect and stroboscopic effect. To overcome these two problems, we introduce a standard gait manifold derived from a training set as an example-based cue and formulate the periodic temporal super resolution as an energy minimization framework considering both reconstruction-based and example-based cues. Experiments with side-view gait videos of 00 subects demonstrate the effectiveness of the proposed method in terms of not only the temporal super resolution quality but also gait-recognition performances.. ) 2),3) 4),5) 6) 7),8) 5 fps ),9) Mori ) Al-Huseiny 9) Level-set 5 fps 2 fps 0) 3) Shechtman 0) The Institute of Scientific and Industrial Research, Osaka University c 202 Information Processing Society of Japan

2 Shimano ) 2 Makihara 2) Akae 3) : 4) ) 2 0),5) Agrawal 5) N N 2),3),6) Xu 7) Ding 8) : Cheng 9) Gaussian Process Latent Variable Model GP-LVM HMMs Scaled GP-LVM 20) Lee 2) Elgammal 22) : Okumura 23) 24) GEI 25) 4) Fourier descriptors 26), Gabor features 27) 2 c 202 Information Processing Society of Japan

3 GSV Fig. Normalized gait silhouette sequence 3. 28) GSV GSV PCA 99% 2 29) P t x(t) x (t + P )=x(t) t () s s s = s P (t) = t P (2) s = s s (3) s P x s(s) =x (s P (s)) (4) s [0, ] 30) x s () = x s (0) N in X in = {x in i }(i =0,...,N in ) f X in, t = {t i }, s P = {s P,i} x in i = x(t i )=x s (s P,i ) (5) t i = t 0 + i f s P,i = s P (t i )=s 0 + i fp X in Q = {x in Q,i} x in Q,i = x s (s Q,i ) (8) s Q,i = s P,i +Δs i (9) {Δs i} s Q = {s Q,i} X in Q s Q x s x s 4.2 x s 29) 3 M N cp {y cp } {scp (= /Ncp )}, ( =0,...,N cp ) [s cp,scp + ] k (k = 0,, 2, 3) a sp,k RM (6) (7) 3 c 202 Information Processing Society of Japan

4 A sp A sp A sp =[a sp sp,0,...,a,3 ]T R 4 M A sp =[A spt 0,...,A sp T N cp ] T R 4N cp M s ŷ( s) ŷ( s) =A spt w( s) (0) w( s) =[0,...,0,,w,w 2,w 3, 0,...,0] T () s scp w = s cp + (s cp scp s s cp + ) (2) w( s) 4 (4 +3) [,w,w 2,w 3 ] 0 w Y cp =[y cp cp 0,...,yN cp ]T A sp 2 2) A sp = DY cp (3) D R 4N cp N cp Y cp (0)(3) s ŷ( s) x s (s; Y cp ) Y cp 4.3 N tr N tr 3) N cp [frame / period] N tr PCA M N tr N cp i Yi tr Yi tr R N cp M z tr i R N cp M PCA PCA M m Y tr {E tr }( =0,,M m ). M m Ŷ tr = Y tr + α E tr (4) =0 α =[α 0,...,α Mm ] T E tr 4.4 0),) Y in Q = {y in Q,i} s Q = {s Q,i } i s Q,i () w(s Q,i) i s Q,i Ŷ (Y cp,s Q,i) =A spt w(s Q,i) =Y cpt D T w(s Q,i) (5) 4 (6) () : ŷ(y cp,s Q,i ) y in Q,i (2) 2 : Y cp Ŷ tr (3) 3 : y s(s; Y cp ) (4) 4 : s Q E(Y cp, α, s Q )= N in Y cpt D T w(s Q,i) y in Q,i 2 + N in i=0 M m λ t N Y cp (Y tr + α E tr cp ) 2 + λ m N cp BY cp 2 + λ s N in N in i= =0 ( s Q,i+ s Q,i P ) 2 (6) B P f P P = fp 4 c 202 Information Processing Society of Japan

5 3 Fig. 3 Iterative solution 2 3 Fig. 2 Effect of the data term and manifold fitness α s Q w 6 3 Akae 3) 3) s init Q = {s init Q,i } P s init Q s 0 Q r Y cp,r α r s r Q. r + s r+ Q Y cp,r α r Taylor E(Y cp, α, s Q ) Y cp α 2 s Q Y cp 2 Y cp,r α r s r Q OU-ISIR Gait Database 32) km 2 60 fps pixel 60 fps M = 2 λ t = λ m =5 λ s = N cp = 00 5 c 202 Information Processing Society of Japan

5.2 9) Morph 3) TSR 4 fps Morph 2 TSR 3 4 60 fps 5 E E = NWH N k=0 W i=0 H [I(i,, k) Î(i,, k)]2 (7) =0 N W H I Î 5 5.

6 5.2 9) Morph 3) TSR 4 fps Morph 2 TSR fps 5 E E = NWH N k=0 W i=0 H [I(i,, k) Î(i,, k)]2 (7) =0 N W H I Î ) Morph 3) TSR NoTSR 24),25) 4) Equal Error Rate EER 33) Rank- 6 7 EER Rank- 4 fps NoTSR 5 fps 4 : fps 2 : Morph 9) 3 : TSR 3) 4 : 5 : 60 fps Fig. 4 Results of temporal super resolution Mean square error Morph TSR 5000 Proposed Fig. 5 Mean squared error of reconstructed images (6) λ t EER Rank- λ t c 202 Information Processing Society of Japan

7 EER Rank- identification rate NoTSR Morph TSR Proposed (a) (b) 6 EER Fig. 6 EER with different features NoTSR Morph TSR Proposed (a) (b) 7 Rank- Fig. 7 Rank- identification rate with different features EER Rank- identification rate NoTSR Morph TSR Proposed NoTSR Morph TSR Proposed λ t 00 λ t 6. Rank- identification rate [fps] 2 [fps] 3 [fps] 4 [fps] 5 [fps] Standard model weight Fig. 8 EER Standard model weight (a) Rank- (b) EER 8 influence of standard manifold weight on EER and Rank- identification rate [fps] 2 [fps] 3 [fps] 4 [fps] 5 [fps] ) Mori, A., Makihara, Y. and Yagi, Y.: Gait Recognition using Period-based Phase Synchronization for Low Frame-rate Videos, Proc. of the 20th Int. Conf. on Pattern Recognition, Istanbul, Turkey, pp (200). 2) Makihara, Y., Tsui, A. and Yagi, Y.: Silhouette Transformation based on Walking Speed for Gait Identification, Proc. of the 23rd IEEE Conf. on Computer Vision and Pattern Recognition, San Francisco, CA, USA (200). 3) Tanawongsuwan, R. and Bobick, A.: A Study of Human Gaits across Different Speeds, Technical report, Georgia Tech (2003). 4) Makihara, Y., Sagawa, R., Mukaigawa, Y., Echigo, T. and Yagi, Y.: Gait Recognition Using a View Transformation Model in the Frequency Domain, Proc. of the 9th European Conf. on Computer Vision, Graz, Austria, pp.5 63 (2006). 5) Wang, Y., Yu, S., Wang, Y. and Tan, T.: Gait Recognition Based on Fusion of Multi-view Gait Sequences, Proc. of the IAPR Int. Conf. on Biometrics 2006, pp (2006). 6) Hossain, M.A., Makihara, Y., Wang, J. and Yagi, Y.: Clothing-Invariant Gait Identification using Part-based Clothing Categorization and Adaptive Weight Control, 7 c 202 Information Processing Society of Japan

8 Pattern Recognition, Vol.43, No.6, pp (200). 7) Sarkar, S., Phillips, J., Liu, Z., Vega, I., Grother, P. and Bowyer, K.: The HumanID Gait Challenge Problem: Data Sets, Performance, and Analysis, Trans. of Pattern Analysis and Machine Intelligence, Vol.27, No.2, pp (2005). 8) Matovski, D., Nixon, M., Mahmoodi, S. and Carter, J.: The effect of time on the performance of gait biometrics, Proc. of the 4th IEEE Int. Conf. on Biometrics: Theory Applications and Systems, Washington D.C., USA, pp. 6 (200). 9) Al-Huseiny, M.S., Mahmoodi, S. and Nixon, M.S.: Gait Learning-Based Regenerative Model: A Level Set Approach, The 20th Int. Conf. on Pattern Recognition, Istanbul, Turkey, pp (200). 0) Shechtman, E., Caspi, Y. and Irani, M.: Space-Time Super-Resolution, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.27, No.4, pp (2005). ) Shimano, M., Okabe, T., Sato, I. and Sato, Y.: Video temporal super-resolution based on self-similarity, Proc. of the 0th Asian Conf. on Computer Vision, Vol., Queeenstown, New zealand, pp (200). 2) Makihara, Y., Mori, A. and Yagi, Y.: Temporal Super Resolution from a Single Quasi-Periodic Image Sequence Based on Phase Registration, Proc. of the 0th Asian Conf. on Computer Vision, Queenstown, New Zealand, pp (200). 3) Akae, N., Makihara, Y. and Yagi, Y.: Gait Recognition using Periodic Temporal Super Resolution for Low Frame-rate Videos, Proc. of the International Joint Conference on Biometrics, Washington, D.C., USA, pp (20). 4) Shahar, O., Faktor, A. and Irani, M.: Space-Time Super-Resolution from a Single Video, Proc. of the IEEE Int. Conf. on Computer Vision and Pattern Recognition, Colorado Springs, USA (20). 5) Agrawal, A., Gupta, M., Veeraraghavan, A. and Narasimhan, S.: Optimal coded sampling for temporal super-resolution, Proc. of of IEEE computer society conference on Computer Vision and Pattern Recognition 200, San Francisco, CA, USA, pp (200). 6) Singh, M., Basu, A. and Mandal, M.K.: Event dynamics based temporal registration, IEEE Trans. on Multimedia, Vol.9, No.5, pp (2007). 7) Xu, X., Wan, L., Liu, X., Wong, T.-T., Wang, L. and Leung, C.-S.: Animating animal motion from still, ACM Trans. Graphics, Vol.27, No.5 (2008). 8) Ding, T., Sznaier, M. and Camps, O.I.: A rank minimization approach to video inpainting, Proc. of the IEEE Int. Conf. on Computer Vision, Rio de Janeiro, Brazil (2007). 9) Cheng, M.-H., Ho, M.-F. and Huang, C.-L.: Gait analysis for human identification through manifold learning and HMM, Pattern Recognition, Vol.4, No.8, pp (2008). 20) Urtasun, R., Fleet, D.J., Hertzmann, A. and Fua, P.: Priors for People Tracking from Small Training Sets, Proc. of of IEEE computer society International conference on Computer Vision 2005, Beiing, China, pp (2005). 2) Lee, C.S. and Elgammal, A.: Coupled Visual and Kinematic Manifold Models for Tracking, International Journal of Computer Vision, Vol.87, No.-2 (200). 22) Elgammal, A. and Lee, C.S.: Separating Style and Content on a Nonlinear Manifold, Proc. of the 2004 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Vol., Washington, DC, USA, pp (2004). 23) Okumura, M., Iwama, H., Makihara, Y. and Yagi, Y.: Performance Evaluation of Vision-based Gait Recognition using a Very Large-scale Gait Database, Proc. IEEE 4th Int. Conf. on Biometrics: Theory, Applications and Systems, Washington, D.C., USA, pp. 6 (200). 24) Liu, Z. and Sarkar, S.: Simplest Representation Yet for Gait Recognition: Averaged Silhouette, Proc. of the 7th Int. Conf. on Pattern Recognition, Vol., pp (2004). 25) Han, J. and Bhanu, B.: Individual Recognition Using Gait Energy Image, Trans. on Pattern Analysis and Machine Intelligence, Vol.28, No.2, pp (2006). 26) Zhao, G., Chen, R., Chen, G. and Li, H.: Recognition of human periodic movements from unstructured information using a motion-based frequency domain approach, Image and Vision Computing, Vol.24, pp (2006). 27) Tao, D., Li, X., Wu, X. and Maybank, S.: Human Carrying Status in Visual Surveillance, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, Vol.2, New York, USA, pp (2006). 28) Makihara, Y. and Yagi, Y.: Silhouette Extraction Based on Iterative Spatio- Temporal Local Color Transformation and Graph-Cut Segmentation, Proc. of the 9th Int. Conf. on Pattern Recognition, Tampa, Florida USA (2008). 29) Murase, H. and Nayar, S.K.: Parametric eigenspace representation for visual learning and recognition, Proc. of SPIE, 203 (993). 30) Murase, H. and Sakai, R.: Moving Obect Recognition in Eigenspace Representation: Gait Analysis and Lip Reading, Pattern Recognition Letters, Vol. 7, pp (996). 3) Oka, R.: Spotting method for classification of real world data, Computer Journal, Vol.4, No.8, pp (998). 32) : OU-ISIR Gait Database, 33) Phillips, P., Moon, H., Rizvi, S. and Rauss, P.: The FERET Evaluation Methodology for Face-Recognition Algorithms, Trans. of Pattern Analysis and Machine Intelligence, Vol.22, No.0, pp (2000). 8 c 202 Information Processing Society of Japan

IPSJ SIG Technical Report 1,a) 1,b) 1,c) 1,d) 2,e) 2,f) 2,g) 1. [1] [2] 2 [3] Osaka Prefecture University 1 1, Gakuencho, Naka, Sakai,

IPSJ SIG Technical Report 1,a) 1,b) 1,c) 1,d) 2,e) 2,f) 2,g) 1. [1] [2] 2 [3] Osaka Prefecture University 1 1, Gakuencho, Naka, Sakai, 1,a) 1,b) 1,c) 1,d) 2,e) 2,f) 2,g) 1. [1] [2] 2 [3] 1 599 8531 1 1 Osaka Prefecture University 1 1, Gakuencho, Naka, Sakai, Osaka 599 8531, Japan 2 565 0871 Osaka University 1 1, Yamadaoka, Suita, Osaka