Fundamentals of Projector-Camera Systems and Their Calibration Methods Takayuki OKATANI To make the images projected by projector s appear as desired, it is e ective and sometimes an only choice to capture the projected images using a camera and control the images of the projectors by analyzing the captured images. To perform this, it is necessary to be able to calibrate the system consisting of projectors and camera s, called the projector-camera system. A projector is similar to a camera geometrically as well as photometrically. Thus, methods of multi-view, projective geometry developed in the field of computer vision, which were originally targeted at cameras, can be utilized for projector-camera systems. This paper explains how to do this, in an unified and exhaustive manner, by categorizing problems in terms of purposes of the system, surface shapes of the projection target etc. Key words: projector-camera, calibration, multi-view geometry 3 1 2 3 4 1 1 1 a 1 b 1 c 1 d 1 980 8579 6 6 01 E-mail: okatani@vision.is.tohoku.ac.jp 457 9
x = [ ] x y x X X = Y Z X O f y (x 0,y 0) Z O X Y 2 3 Y Z 1 1. 1. 1 1 2 3 O-XYZ O -X Y Z o-xy Z X, Y x, y x, y x, y XGA x, y 0, 0 1024, 768 x, y X, Y, Z 458 10 x X y Y K R t 1 1 Z 1 X, Ỹ, Z x X Z y Ỹ Z K 3 3 R 3 3 t 3 R t 2 3 4 K f sf x 0 K 0 αf y0 2 0 0 1 f s a x 0, y 0 f s 0 a a 1x 0, y 0 2 x 0, y 0 x 0, y 0 y 0 x 0
1 OpenCV 1 1. 2 photometric 2 3 I p W m 2 E o 3 E o E o g I p 3 g I p 3 E o cos a E o g I p 4 r 2 cos q a q a q r θ ~ 3 E o E o BRDF BRDF l p ṽ 4 2 BRDF f o l p, ṽ ṽ L o L o f o l p, ṽ E o 5 4 5 I p ṽ I p BRDF 2. 4 α ~ 1 http://doc.opencv.org 459 11
? 4 or? 2. 1 1 x, y X, Y, Z 1 O-XYZ XY O Z Z 0 X, Y, 0 Z 0 1 x X X y K r 1 r 2 t Y H Y 6 1 1 1 r 1, r 2 R 1 2 H K r 1 r 2 t X, Y x, y x, y, 1 H X, Y, 1 X, Y, 1 6 x H H 1 x 7 x x, y, 1, x x, y, 1 1 2 H H 1 x Hx x, x 4 x i, x i i 1, OpenCV findhomography 2. 2 x x 1 460 12 5 3 3 F x Fx 0 8 F K, K F K EK 1 9 E E t R t t 3 3 t a t a R, t X RX t F x, x 7 8 OpenCV findfundmentalmat E 5 3. 5 3 1 1 1 3. 1
6 b x, y u, v u u x, y, v v x, y 4 u x, y, v x, y 3. 2 R, t 3. 2. 1 2.1 H c H p H pc H c H 1 p H cp H 1 pc 1 H cp H p H c 10 H p X, Y x, y x, y, 1 H p X, Y, 1 4 H cp H p H c 6 X, Y x c, y c 4 2.1 H c H p H cp H c H p H c 6 a 4 5 1 2 H c 6 5 1 4 H c 4 3. 2. 2 2 6 c K K 2.2 x x E K x E Kx 0 E t R R, t 4 461 13
1 1 F F R, t 1 1 y 3. 3 K 3 7 H c 4 H 1 c 3 7 f 462 14 x 0, y 0 0 4. 1 4. 1 n 2 n n 2 x y n
7 Brugel 7 1 7 1 x y 4. 2 1 1 Q AR SIFT 8 OpenCV 2 1 SIFT SIFT RANSAC RANdom SAmple Consensus 2 9 10 1 R. Hartley and A. Zisserman: Multi-View Geometry in Computer Vision Cambridge University Press, 2000. 2 B. K. P. Horn: Robot Vision MIT Press, Cambridge, 1986. 3 1993 4 T. Okatani and K. Deguchi: Easy calibration of a multi-projector display system, Int. J. Comput. Vision, 85 2009 1 18. 5 D. Abe, T. Okatani and K. Deguchi: Flexible online calibration for a mobile projector-camera system, Proc. ACCV 2010, Lect. Notes Comput. Sci., 4695 2011 565 579. 6 T. Okatani and K. Deguchi: Autocalibration of a projectorcamera system, IEEE Trans. Pattern Anal. Machine Intell., 27 2005 1845 1855. 7 Z. Zhang: A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Machine Intell., 22 2000 1330 1334. 8 David G. Lowe: Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vision, 60 2003 91 110. 9 3 2010 10 M. Ashdown, T. Okabe, I. Sato and Y. Sato: Robust contentdependent photometric projector compensation, Proc. IEEE Int. Workshop on Projector-Camera Systems 2006 pp. 60 67. 2014 6 19 2 http://docs.opencv.org/doc/tutorials/tutorials.html 2D Features framework 463 15