情報処理学会研究報告 IPSJ SIG Technical Report Vol.2014-CVIM-190 No /1/24 RGB-D 1 1 *1 Relighting with Dynamic Light Environments Using an RGB-D Camera

Size: px

Start display at page:

Download "情報処理学会研究報告 IPSJ SIG Technical Report Vol.2014-CVIM-190 No /1/24 RGB-D *1 *1 *1 Relighting with Dynamic Light Environments Using an RGB-D Camera"

かねろういせき
5 years ago
Views:

1 RGB-D *1 *1 *1 Relighting with Dynamic Light Environments Using an RGB-D Camera Takuya Ikeda *1 Francois de Sorbier *1 and Saito Hideo *1 Abstract RGB-D GPU 10fps Keywords : Relighting, Lambertian model, RGB-D camera, GPU 1 (Relighting) CG [1] Augmented Reality AR AR [2] 3 Src Dst *1 *1 Keio University Dst Dst Zhen [3] 1 3 Zhen Lambertian Zhen Lambertian Debevec [4] Wenger [5] Light Stage Reflectance Functions 3 Wenger Light Stage 1

2 (Src) (Src) (Src) (Dst) (Dst) SH [10] SH [12] [3] 1 Fig. 1 Proposed relighting flow. RGB-D RGB-D 30fps Dst AR 2 1 RGB-D 1 1 Src Dst Dst SH Src Dst Dst GPU 2.1 Chen [6] ( 2 (a)) Canny n n 1, 0 M c M d M f 2 M f = M d ANDM c (1) 2

1 n c n d n c < n d 2(b) 2 Chen [6] Chen region growing k-means Chen

(i(u) i(v))d t 1 (v) (2) g s g c σ s, σ c Ω s u n s u v i(u) i(v) D t 1

Ω s 2 (2) (2) t D t D m 2(c) (d) RGB-D 2(e) (f) (a) (d) (b) (e) 2 Fig.

2 K V (u) = D m K 1 [u, 1] 3 V RGB-D OpenNI [8] D m D b (u) = 1 k (u) v Ω

3 1 n c n d n c < n d 2(b) 2 Chen [6] Chen region growing k-means Chen k-means GPU [7] RGB-D D D t (u) = 1 k(u) v Ω s,d t 1 / 0 g s (u v)g c (i(u) i(v))d t 1 (v) (2) g s g c σ s, σ c Ω s u n s u v i(u) i(v) D t 1 (v) v k ( u) k(u) = v Ω s,d t 1 / 0 g s(u v)g c (i(u) i(v)) k-means u u v Ω s 2 (2) (2) t D t D m 2(c) (d) RGB-D 2(e) (f) (a) (d) (b) (e) 2 Fig. 2 Depth map modification. (c) (f) K V (u) = D m K 1 [u, 1] 3 V RGB-D OpenNI [8] D m D b (u) = 1 k (u) v Ω s g s (u v)g d (D m (u) D m (v))d m (v) (3) D b g s g d σ s, σ d Ω s u n s D m (u) D m (v) k ( u) Stefan [9] Stefan 3 9 3

3 Stefan 2 2 u = (u, v) N(u) N(u) = (V (u+1, v) V (u, v)) (V (u, v+1) V (u, v)) N(u) 2.3 cube map cube map 3(a) 3(b) 32 32 6( ) cube map cube map cube map cube map 2.

θ ϕ L(ω) max(cos θ, 0) SH [10] Ramamoorthi [11] (4) SH E(x) = l l=0 m= l A l (θ)l lm Y lm (ω) (5) Y lm (ω) SH A l (θ) L lm max(cos θ, 0) L(ω) SH 2.

4 3 Stefan 2 2 u = (u, v) N(u) N(u) = (V (u+1, v) V (u, v)) (V (u, v+1) V (u, v)) N(u) 2.3 cube map cube map 3(a) 3(b) ( ) cube map cube map cube map cube map 2.4 Zhen [3] 3 x E(x) E(x) = L(ω)max(cos θ, 0)dω (4) Ω L(ω) x ω = (θ, ϕ) max(cos θ, 0) dω x (a) 3 Fig. 3 (b) cube map cube map. Camera and cube map coordinates. θ ϕ L(ω) max(cos θ, 0) SH [10] Ramamoorthi [11] (4) SH E(x) = l l=0 m= l A l (θ)l lm Y lm (ω) (5) Y lm (ω) SH A l (θ) L lm max(cos θ, 0) L(ω) SH 2.3 SH Ramamoorthi [11] ronbun 9 SH l = 2 9 SH max(cos θ, 0) θ = 0 A std l = A l (0) A std l SH [12] A l (θ) Lambertian Src Dst i dst (u) = i src (u) Esrc (x) E dst (x) (6) i dst (u) u i src (u) E src (x) Src L src lm E dst (x) Dst L dst lm Dst 3 Src 4 Src 4

5 4 Src Fig. 4 The Object and Src light environment using our experiment. Human 3(b) cube map D Microsoft Kinect RGB RGB RGB- RGB- D OpenNI [8] 5cm 1 GPU GPU kmeans SH GPU [7] [12] [13] OS: Windows7 CPU: Intel Core i7-3940xm 3.00GHz RAM: 32.0GB GPU: NVIDIA GeForce GTX 680M :Microsoft Visual C Table 1 Parameters of experiment. Human n d 10 n c 8 [6] σ s 1.4 σ c 1.2 n s 5 t 40 k-means Cluster number 20 Iteration 20 σ s 150 σ d 60 n s 13 Dst Night Road Dst cube map 2 1 st 49 th 159 th 222 th 3 k-means CPU GPU 9 Human 10.20fps 2 3 Computation time. Human [9] SH [12] 0.01 [3] (Human) Src (z+ ) 4 RGB-D 5

1st frame 159th frame 49th frame 222th frame 図 5 リライティング結果 (Human) Fig. 5 Relighting results of Human.

edu/lightstages/ [2] Paul Debevec Rendering synthetic objects into real scenes: bridging traditional and image-ased graphics with global illumination and high dynamic range photography, SIGGRAPH, pp.

[4] Paul Debevec, Tim Hawkins, Chris Tchou and Haarm-Pieter Duiker, Westley Sarokin and Mark Sagar Acquiring the reﬂectance ﬁeld of a human face, SIGGRAPH, pp. 145-156, 2000.

[11] [12] [13] mation with time-multiplexed illumination ACM Transactions on Graphics 24, 3, pp.756 764, 2005.

6 1st frame 159th frame 49th frame 222th frame 図 5 リライティング結果 (Human) Fig. 5 Relighting results of Human. としたまた提案手法の処理を GPU による並列演算で高速化する事により動的環境に対して約 10fps の速度でリライティングを行えることを示した今後 [6] の課題として鏡面反射を含む陰影や影を考慮したモデルを用いたリライティングの精度向上を目指す参考文献 [1] [2] Paul Debevec Rendering synthetic objects into real scenes: bridging traditional and image-ased graphics with global illumination and high dynamic range photography, SIGGRAPH, pp , [3] Zhen Wen, Zicheng Liu and Thomas S Huang Face Relighting with Radiance Environment Maps, Computer Vision and Pattern Recognition, pp , [4] Paul Debevec, Tim Hawkins, Chris Tchou and Haarm-Pieter Duiker, Westley Sarokin and Mark Sagar Acquiring the reﬂectance ﬁeld of a human face, SIGGRAPH, pp , [5] Andreas Wenger, Andrew Gardner, Chris Tchou, Jonas Unger, Tim Hawkins and Paul Debevec Performance relighting and reﬂectance transfor 2014 Information Processing Society of Japan [7] [8] [9] [10] [11] [12] [13] mation with time-multiplexed illumination ACM Transactions on Graphics 24, 3, pp , Li Chen, Hui Lin, and Shutao Li, Depth image enhancement for Kinect using region growing and bilateral ﬁlter, International Conference on Pattern Recognition, pp , J. Sirotkovic, H. Dujmic, V. Papic, K-Means Image Segmentation on Massively Parallel GPU Architecture, MIPRO, pp , OpenNI, Stefan Holzer, Radu Bogdan Rusu, M. Dixon, Suat Gedikli, and Nassir Navab, Adaptive neighborhood selection for real-time surface normal estimation from organized point cloud data using integral images, IROS, pp , Sloan Peter-Pike, Stupid Spherical Harmonic (SH) Tricks, Game Developer s Conference, 2008 R. Ramamoorthi, P. Hanrahan, An eﬃcient representation for irradiance environment maps, SIGGRAPH, pp , Derek Nowrouzezahrai, Patricio Simari, Eugene Fiume, Sparse Zonal Harmonic Factorization for Eﬃcient SH Rotation, ACM Transactions on Graphic, 31(3): 23, B. Bilgic, B.K.P. Horn, I. Masaki, Eﬃcient Integral Image Computation on the GPU,In IEEE Intelligent Vehicles Symposium, pp ,

3 3 3 Knecht (2-3fps) AR [3] 2. 2 Debevec High Dynamic Range( HDR) [4] HDR Derek [5] 2. 3 [6] 3. [6] x E(x) E(x) = 2π π 2 V (x, θ i, ϕ i )L(θ

3 3 3 Knecht (2-3fps) AR [3] 2. 2 Debevec High Dynamic Range( HDR) [4] HDR Derek [5] 2. 3 [6] 3. [6] x E(x) E(x) = 2π π 2 V (x, θ i, ϕ i )L(θ (MIRU212) 212 8 RGB-D 223 8522 3 14 1 E-mail: {ikeda,charmie,saito}@hvrl.ics.keio.ac.jp, sugimoto@ics.keio.ac.jp RGB-D Lambert RGB-D 1. Augmented Reality AR [1] AR AR 2 [2], [3] [4], [5] [6] RGB-D RGB-D