(a) (b) (c) Fig. 2 2 (a) ; (b) ; (c) (a)configuration of the proposed system; (b)processing flow of the system; (c)the system in use 1 GPGPU (

1 1 1 (a) (b) imperceptible A Realtime and Adaptive Technique for Projection onto Non-Flat Surfaces Using a Mobile Projector Camera System Eiji Seki, 1 Dao Vinh Ninh 1 and Masanori Sugimoto 1 In this paper, we describe a realtime and adaptive technique for projection onto non-flat surfaces using a mobile projector camera system. The proposed technique allows a user to rapidly capture the 3D geometry of an object by using an imperceptible checkerboard pattern and project a geometrically calibrated image onto surfaces of the captured object. The system can always show calibrated images to a given user viewpoint while it is moved or rotated. 1 : (a) ; (b) Fig. 1 Projection in a car: (a)non-calibrated image; (b)calibrated image 1. 6)2)11)5) 1(a) 1(b) 7)9)8) 3 3 CPU GPGPU General-Purpose computing on Graphics Processing Unit GPGPU 1 The University of Tokyo 1 c 2011 Information Processing Society of Japan

(a) (b) (c) Fig. 2 2 (a) ; (b) ; (c) (a)configuration of the proposed system; (b)processing flow of the system; (c)the system in use 1 GPGPU 2. 2.1 2(a) DLP ACER P3251 PointGrey Firefly MV 2(b) imperceptible 3 ICP 1 GPGPU PowerVR SGX Imagination Technologies 12) CUDA NVIDIA GPGPU C NVIDIA Tegra NVIDIA 10) 3 3 Fig. 3 Flow of 3D acquisition 2(c) 1 2.2 3 3 4) 3 3 2 c 2011 Information Processing Society of Japan

(a) (b) (c) (a) 4 1 : (a) ; (b) Fig. 4 Image calibration for projection onto flat surfaces: (a)a plane, a projector and a view point;(b)correspondence between a pixel on a projector and that from a view point (b) 5 : (a) ; (b) ; (c) Fig. 5 Image calibration for projection onto non-flat surfaces : (a)a non-flat surface, a projector, and a view point; (b)correspondence between triangle on a projector and that from a view point; (c)a calibrated image to be projected onto a non-flat surface 2.3 ICP 2(c) ICP Iterative Closest Point 1) ICP 2.4 3 2.4.1 1 1 1 Homography p a = K a H ba K b p b (1) 1 p b p a K a K b H ba 2 H ba = R t nt d R t 2 n d S S Homography 4 2.5 1 3 (2) 3 c 2011 Information Processing Society of Japan

3. GPGPU (a) (b) (c) 6 : (a) ; (b) ; (c) Fig. 6 Embedding structured light pattern in image: (a)an original image; (b)a pattern embeded image; (b)captured image from camera 1 1 2 2 1 5(b) 5(c) 2.6 3 Cotting 3) DLP 6(a) 6(b) 6(c) 3.1 GPGPU GPGPU GPGPU GPGPU ( 1 ) ( 2 ) ( 3 ) GPGPU GPGPU 1 2 3 3 3.2 GPGPU CPU Intel Core i7 860 2.80 GHz 1800 640 480 800 600 ICP 41%, 3 25% 11% GPGPU ICP 3.3 GPGPU 3.3.1 3 3 43% 4 14% 2 GPGPU GPGPU 640 480 4 c 2011 Information Processing Society of Japan

1 GPGPU Table 1 Results of applying a GPGPU to feature point detection and mask processes GPU CPU ms GPU ms 0.4 10.5 1.6 4 3.4 1.0 13.9 3.0 2 GPGPU Table 2 Results of applying a GPGPU to image calibration CPU ms GPU ms 1.58 11.22 3.09 11.22 4.67 8 CPU 7 Fig. 8 Finding a mapping between each pixel Fig. 7 Pixel-by-pixel processing and triangle is conducted by a CPU is needed to find a corresponding triangle GPGPU 3.3.2 3 3 GPGPU 7 1 1 2 GPGPU CPU GPU GPGPU 1 8 2 1 GPGPU 1 CPU 2 Homography GPGPU 1 CPU 2 GPU 4. 4.1 GPGPU 4.1.1 3 3 GPGPU GPGPU CPU CPU 3.2 GPU NVIDIA GeForce GTX 295 SP 240 2 1.24GHz 4.1.2 3 GPGPU 1 1 GPU 1 4.6 10.9 1 2 2 1 SP 1 5 c 2011 Information Processing Society of Japan

(a) (b) 9 ; (a)(b) Fig. 9 Projecting images onto non-flat surfaces; non-calibrated images (left) and calibrated images (right) in (a) and (b), respectively GPGPU 4.1.3 3 GPGPU 3.3.2 CPU GPGPU CPU 2 CPU GPGPU 2.4 4.2 9 5. PC GPGPU ICP GPGPU 1) Besl, P. and McKay, N.: A Method for Registration of 3-D Shapes, IEEE Trans. PAMI, Vol.14, No.2, pp.239 256 (1992). 2) Cao, X., Forlines, C. and Balakrishnan, R.: Multiuser Interaction Using Handheld Projectors, Proceedings of ACM UIST 2007, pp.43 52 (2007). 3) Cotting, D., Naef, M., Gross, M. and Fuchs, H.: Embedding Imperceptible Patterns into Projected Images for Simultaneous Acquisition and Display, Proceedings of IEEE/ACM ISMAR 2004, pp.100 109 (2004). 4) Dao, V. and Sugimoto, M.: A Dynamic Geometry Reconstruction Technique for Mobile Devices Using Adaptive Checkerboard Recognition and Epipolar Geometry, IEICE Trans. Information and Systems, Vol.E94-D, No.2, pp.336 348 (2011). 5) Löchtefeld, M., Gehring, S., Schöning, J. and Krüger, A.: ShelfTorchlight: Augmenting a Shelf using a Camera Projector Unit, Proceedings of Workshop on Ubiprojection 2010 (2010). retrieved at http://eis.comp.lancs.ac.uk/workshops/ubiproject 2010/pdf/loechterfeld ubiprojection2010.pdf. 6) Mistry, P., Maes, P. and Chang, L.: WUW - Wear Ur World: A Wearable Gestural Interface, Proceedings of ACM CHI 2009, pp.4111 4116 (2009). 7) Raskar, R., Baar, R., Beardsley, P., Willwacher, T., Rao, S. and Forlines, C.: ilamps: Geometrically Aware and Self-Configuring Projectors, Proceedings of ACM SIGGRAPH 2003, pp.809 818 (2003). 8) Zollmann, S. and Bimber, O.: Imperceptible Calibration for Radiometric Compensation, Proceedings of EUROGRAPHICS 2007, pp.61 64 (2007). 9) Zollmann, S., Langlotz, T. and Bimber, O.: Passive-Active Geometric Calibration for View-Dependent Projections onto Arbitrary Surfaces, Journal of Virtual Reality and Broadcasting, Vol.4, No.6 (2007). 10) CUDA CUDA Everywhere x86 ARM NVIDIA Impress http://pc.watch.impress.co.jp/docs/column/ubiq/20100924 395960.html 2011-02-14. 11) CoGAME: Vol.12, No.3, pp.285 294 (2007). 12) CES GPGPU, Imagination Technologies, http://techon.nikkeibp.co.jp/article/news/20090111/163928/ 2011-02-14. 6 c 2011 Information Processing Society of Japan