3 1 2 3 3 3 Tutorial: Active 3D reconstruction of moving objects HIROSHI KAWASAKI, 1 RYUSUKE SAGAWA 2 and RYO FURUKAWA 3 Recently, 3D scanning devices which can capture moving objects are now widely available even in supermarket. Although such devices can retrieve depth information of the scene almost in realtime, accuracy, resolution and fps are significantly different each other. Indeed, 3D reconstruction algorithm for each device is also different and we still have no idea which one becomes most popular in the near future. In this tutorial, we introduce basic algorithm of current 3D scanning devices as well as new technology proposed recently. 1. 3 1 Kagoshima University 2 National Institute of Advanced Industrial Science and Technology 3 Hiroshima City University 2. 3 1 2 TOF (time-of-flight) TOF 6),24) 30FPS 1 c 2011 Information Processing Society of Japan
1 プロジェクタ グレイコードシーケンス ( ) ( ). 2.1 3 3 ( ) ( 35) ) ( 1) 3 2 3 ( 1 ) ( 2 ) 3 ( 3 ) 3 2 (44) ) 43) 3 4),19) Matlab 19) 2 2.2 3 3( ) 1 1 3) 2 c 2011 Information Processing Society of Japan
Target object camera??? projector 1 0 1 0 1 0 0 1 0 1 0 1 0 3 ( ) ( ) ( ). 2 ( ),( ), ( ) ( x ), ( ). 2 2.2.1 1),5),7),12),15) 3 27),31),39) 3 2.2.2 16),29),33),36) () 4 3. DLP 3 c 2011 Information Processing Society of Japan
5 3 4 (a) (a) 4 (b) 3.1 Hall-Holt Rusinkiewicz 14),31) 4(a) 4 2 4 110 110 DLP 30 60Hz 60FPS 4 4(b) (b) 3.2 2 3 Weise 39) DLP DLP RGBW RGB 120Hz 120FPS 5 12) 13) Weise 4 c 2011 Information Processing Society of Japan
6 (a) (b) (c) Narasimhan 27) (a)dmd (b) DMD (c) 2/3π 3 2/3π 2/3π 3.3 DMD DLP 120Hz DMD(Digital Micromirror Device) Narasimhan 27) DMD 6(a) 0 255 300 3000FPS 6(b) DMD 6(c) 0 255 DLP DMD 160 20 1000FPS 20 20msec 300FPS 4. 4.1 1 1 7(a) 5 c 2011 Information Processing Society of Japan
7 カメラ画像面 ラインパターン (3 個組 ) の対応 対応点 エピポーラ線の対応 (a) カメラ画像面 のエピポーラ線 プロジェクタ画像面 のエピポーラ線 の対応点候補 のエピポーラ線 (c) カメラ画像面 プロジェクタ画像面 特徴点間の対応 エピポーラ線 (b) プロジェクタ画像面 (a) (b) (c) 2 7(b) 2 2 3 7(c) 4.2 2 1 1 1 2 1 2 9) 3 4.3 1 1 6 c 2011 Information Processing Society of Japan
8 ( ) ( ) ( 3) ) ( )Koninckx. 3 2 4.4 4.4.1 2),22),23),34),36),42) ID 36) RGB 34) 42) 9 ( )M-array Murano 26) ( )2*2 Cell Kimura 20) ( )Kinect 30). On/Off 23) 2) Koninckx 21) ( ) ID 22) 4.4.2 On/Off 7 c 2011 Information Processing Society of Japan
10 ( ) ( ) ( )2. 16),20),26),29),33),38) Morano M-array M*M 26) 2*2 RGB 20) Kinect 25) 2 4.4.3 M-array 17),32),37) 1 10) 3 17) 1000fps 32) 37) 2 18) 5. 8),40),41) 41).,. 8),41) 6. 3 3 8 c 2011 Information Processing Society of Japan
6.1 3 11 12 6.2 3 3 3 3 ( 13) 11) 17),32),37) 6.3 CG Kinect (a)frame No. 809/1000 fps. (b)frame No. 824/1000 fps. 11 frame no. 12/300fps. frame no. 19/300fps. 12 Results of a exploding balloon. Results of a breaking dish. 3 32) 28) 9 c 2011 Information Processing Society of Japan
Camera 2 Projector 3 Camera 3 Projector 4 Projector 2 13 7. Camera 1 Target object Camera 4 Projector 1 ( 11) ): ( ) ( ) ( ) 3 TOF (LR030) (SCOPE)ICT (101710002) 21700183,21200002 1) Abdul-Rahman, H.S., Gdeisat, M.A., Burton, D.R., Lalor, M.J., Lilley, F. and Moore, C.J.: Fast and robust three-dimensional best path phase unwrapping algorithm, Appl. Opt., Vol.46, No.2, pp.6623 6635 (2007). 2) Artec: United States Patent Application 2009005924 (2007j). 3) Batlle, J., Mouaddib, E. and Salvi, J.: Recent progress in coded structured light as a technique to solve the correspondence problem: a survey, Pattern Recognition, Vol.31, No.7, pp. 963 982 (1998). 4) Bouguet, J.-Y.: Camera Calibration Toolbox for Matlab. http://www.vision.caltech.edu/ bouguetj/calib doc/. 5) Boyer, K.L. and Kak, A.C.: Color-encoded structured light for rapid active ranging, IEEE Trans. on PAMI, Vol.9, No.1, pp.14 28 (1987). 6) Canesta, Inc.: CanestaVision EP Development Kit. http://www.canesta.com/devkit.htm. 7) Caspi, D., Kiryati, N. and Shamir, J.: Range imaging with adaptive color structured light, IEEE Trans. on PAMI, Vol.20, No.5, pp.470 480 (1998). 8) Davis, J., Nehab, D., Ramamoorthi, R. and Rusinkiewicz, S.: Spacetime Stereo: A Unifying Framework for Depth from Triangulation, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol.27, No.2, pp.296 302 (2005). 9) Furukawa, R. and Kawasaki, H.: Uncalibrated multiple image stereo system with arbitrarily movable camera and projector for wide range scanning, IEEE Conf. 3DIM, pp.302 309 (2005). 10) Furukawa, R. and Kawasaki, H.: Self-calibration of Multiple Laser Planes for 3D Scene Reconstruction, 3DPVT, pp.200 207 (2006). 11) Furukawa, R., Sagawa, R., Kawasaki, H., Sakashita, K., Yagi, Y. and Asada, N.: One-shot Entire Shape Acquisition Method Using Multiple Projectors and Cameras, 4th Pacific-Rim Symposium on Image and Video Technology, IEEE Computer Society, pp.107 114 (2010). 12) Ghiglia, D.C. and Pritt, M.D.: Two-Dimensional Phase Unwrapping: Theory, Algorithms, And Software, Wiley-interscience (1998). 13) Gühring, J.: Dense 3-d surface acquisition by structured light using off-the-shelf components, Videometrics and Optical Methods for 3D Shape Measurement, Vol.4309, pp.220 231 (2001). 14) Hall-Holt, O. and Rusinkiewicz, S.: Stripe Boundary Codes for Real-Time Structured-Light Range Scanning of Moving Objects, ICCV, Vol.2, pp.359 366 (2001). 15) Inokuchi, S., Sato, K. and Matsuda, F.: Range imaging system for 3-D object recognition, ICPR, pp.806 808 (1984). 16) Je, C., Lee, S.W. and Park, R.-H.: High-Contrast Color-Stripe Pattern for Rapid Structured- Light Range Imaging, ECCV, Vol.1, pp.95 107 (2004). 17) Kawasaki, H., Furukawa, R.,, Sagawa, R. and Yagi, Y.: Dynamic scene shape reconstruction using a single structured light pattern, CVPR, pp.1 8 (2008). 18) Kawasaki, H., Furukawa, R., Sagawa, R., Ohta, Y., Sakashita, K., Zushi, R., Yagi, Y. and Asada, N.: Linear solution for oneshot active 3D reconstruction using two projectors, 3DPVT (2010). 19) KawasakiLab: Projector Calibration Toolbox. http://www.ibe.kagoshimau.ac.jp/ cgv/research/projcalib.html. 20) Kimura, M., Mochimaru, M. and Kanade, T.: Measurement of 3D Foot Shape Deformation in Motion, 4th Digital Human Symposium (2009). 21) Koninckx, T.P., Geys, I., Jaeggli, T. and Gool, L. J.V.: A Graph Cut based Adaptive Structured Light Approach for Real-Time Range Acquisition, 3DPVT, pp.413 421 (2004). 22) Koninckx, T.P. and Gool, L.V.: Real-Time Range Acquisition by Adaptive Structured Light, 10 c 2011 Information Processing Society of Japan
IEEE Trans. on PAMI, Vol.28, No.3, pp.432 445 (2006). 23) Maruyama, M. and Abe, S.: Range sensing by projecting multiple slits with random cuts, SPIE Optics, Illumination, and Image Sensing for Machine Vision IV, Vol.1194, pp.216 224 (1989). 24) Mesa Imaging AG.: SwissRanger SR-4000. http://www.swissranger.ch/index.php. 25) Microsoft: Xbox 360 Kinect. http://www.xbox.com/en-us/kinect. 26) Morano, R.A., Ozturk, C., Conn, R., Dubin, S., Zietz, S. and Nissanov, J.: Structured light using pseudorandom codes, IEEE Trans. on PAMI, Vol.20, No.3, pp.322 327 (1998). 27) Narasimhan, S.G., Koppal, S.J., and Yamazaki, S.: Temporal Dithering of Illumination for Fast Active Vision, Proc. European Conference on Computer Vision, pp.830 844 (2008). 28) Ogawara, K., Furukawa, R., Sagawa, R. and Kawasaki, H.: Marker-less Motion Capture using Dense Human-body Shape Scanning System, 3DPVT (2011). 29) Pan, J., Huang, P.S. and Chiang, F.-P.: Color-coded binary fringe projection technique for 3-D shape measurement, Optical Engineering, Vol.44, No.2, pp.23606 23615 (2005). 30) Primesense: United States Patent Application US 2010/0118123 (2010j). 31) Rusinkiewicz, S., Hall-Holt, O. and Levoy, M.: Real-Time 3D Model Acquisition, Proc. SIGGRAPH, pp.438 446 (2002). 32) Sagawa, R., Ota, Y., Yagi, Y., Furukawa, R., Asada, N. and Kawasaki, H.: Dense 3D reconstruction method using a single pattern for fast moving object, ICCV (2009). 33) Salvi, J., Batlle, J. and Mouaddib, E.M.: A robust-coded pattern projection for dynamic 3D scene measurement, Pattern Recognition, Vol.19, No.11, pp.1055 1065 (1998). 34) Sato, K. and Inokuchi, S.: Range-Imaging System Utilizing Nematic Liquid Crystal Mask, Proc. Int. Conf. on Computer Vision, pp.657 661 (1987). 35) Song, Z. and Chung, R.: Novel Method for Structured Light System Calibration, IEEE Transactions on Instrumentation and Measurement, Vol.57, No.11 (2010). 36) Tajima, J. and Iwakawa, M.: 3-D data acquisition by rainbow range finder, ICPR, pp.309 313 (1990). 37) Ulusoy, A.O., Calakli, F. and Taubin, G.: One-Shot Scanning using De Bruijn Spaced Grids, The 7th IEEE Conf. 3DIM (2009). 38) Vuylsteke, P. and Oosterlinck, A.: Range Image Acquisition with a Single Binary-Encoded Light Pattern, IEEE Trans. Pattern Anal. Mach. Intell., Vol.12, No.2, pp.148 164 (1990). 39) Weise, T., Leibe, B. and Gool, L.V.: Fast 3D Scanning with Automatic Motion Compensation, Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp.1 8 (2007). 40) Young, M., Beeson, E., Davis, J., Rusinkiewicz, S. and Ramamoorthi, R.: Viewpoint- Coded Structured Light, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2007). 41) Zhang, L., Snavely, N., Curless, B. and Seitz, S. M.: Spacetime Faces: High-Resolution Capture for Modeling and Animation, ACM Annual Conference on Computer Graphics, pp. 548 558 (2004). 42) Zhang, S. and Huang, P.: High-resolution, Real-time 3D Shape Acquisition, Proc. Conference on Computer Vision and Pattern Recognition Workshop, p.28 (2004). 43) Zhang, S. and Haung, P.S.: Novel Method for Structured Light System Calibration, Optical Engineering, Vol.45, No.8 (2006). 44), CVIM 2002-33-1 IEEE Computer Society, pp.1 8 (2002). 11 c 2011 Information Processing Society of Japan