Vol. 52 No. 2 901 909 (Feb. 2011) Gradient-Domain Image Editing is a useful technique to do various-type image editing, for example, Poisson Image Editing which can do seamless image composition. This paper presents Gradient-Domain Image Filtering based on Gradient-Domain Editing. The proposed filtering process is carried out by the following computational scheme. Firstly gradient space is divided, for example, according to gradient strength. New images, corresponding to the strong gradient and the weak gradient, are reconstructed by solving the Poisson equations. The reconstructed strong-gradient image is a flattened image (like a illustration image) and the weak-gradient image is a gradation-extraction image. By mixing the gradients with appropriate weights and solving the corresponding Poisson equation, we can get various filtering effects. To do this efficiently and interactively, we propose a technique to generate the fundamental images from the divided gradients and blend the images in image space without solving Poisson equations. Furthermore, by devising how to divide gradient space, we show that various-type filters are constructed, for example, gradation enhancement by the direction, line extraction/deletion, and contrast-preserving brightness enhancement. The last one is carried out by blending the fundamental images optimally based on a training image. 1 Poisson Poisson Poisson Gradient-domain Image Filtering Shinichiro Miyaoka 1 1. 2003 SIGGRAPH Poisson Image Editing 1) Poisson Image Editing Poisson 2),3) 4),5),12) HDR High Dynamic Range 6) Poisson HDR 2007 ICCV 7) Poisson 8),9) 1 Diffusion Curves 8) Poisson Mean-Value Cloning 1 School of Media Science, Tokyo University of Technology 901 c 2011 Information Processing Society of Japan
902 10) Poisson Poisson 3 Poisson 2. f s g g g Poisson g f Δf = g over Ω (1) Δf = 2 f x 2 + 2 f y 2 g =(g x,g y), g = gx x + gy y Ω Neumann 6) Dirichlet 1) Poisson Image Editing 1) 1 1 Fig. 1 Poisson Image Editing Image composition by Poisson Image Editing. f s f t Ω Ω Poisson Poisson Dirichlet Ω Ω Δf =Δf s over Ω f Ω = f t Ω (2) Poisson SOR 11) SOR f n+1 (x, y) =(1 ω)f n (x, y)+ω(f n+1 (x, y 1) + f n+1 (x 1,y) + f n (x +1,y)+f n (x, y +1) Δf s(x, y))/4 n ω ω 1 2 1.95 3. 3.1 2 (3)
903 2 Fig. 2 Separation to strong edge and gradation. g g st g wk th RGB { g if g th g st = 0 else { (4) g if g <th g wk = 0 else Ω 1 3 (a) 0 3(a) 20 Neumann 0 Poisson 3(b) 0 Dirichlet 40 Dirichlet 0 Dirichlet 0 Poisson (1) 2 g 1) 3 Fig. 3 Boundary conditions. 4 Fig. 4 Strong-gradient image and weak-gradient image. Poisson g st f st g wk f wk 4 256 th 20 4 5 3.2 α β (5) α = β =1
904 5 Fig. 5 Divided gradients. 6 Fig. 6 Fundamental images. Δf = (αg st + βg wk )overω f Ω = f s Ω (5) Poisson PC α β Poisson Poisson f 0 f st f wk f st f wk 0 3.1 f st f wk Fig. 7 7 Blending results of fundamental images. Δf 0 =0overΩ, f 0 Ω = f s Ω Δf st = g st over Ω, f st Ω =0 (6) Δf wk = g wk over Ω, f wk Ω =0 f = f 0 + αf st + βf wk (7) 4(a) 6 6(a) 6(b) (c) 0 6 7 7(a) 2 0 7(b) 2 1 7(c) 1 2
905 n g = g i (8) i=1 Δf 0 =0overΩ, f 0 Ω = f s Ω Δf i = g i over Ω, f i Ω =0 (i =1 n) (9) n g = α ig i (10) i=1 Poisson n f = f 0 + α if i (11) i=1 4. 4.1 e (12) 8 (a) g wk e g dir1 g dir2 e (, ) { g dir1 =(g wk, e)e g wk ( ) (12) g dir2 = g wk g dir1 8 Fig. 8 Gradation enhancement according to direction. x 8 8(c) x 8(d) y 3 0 8(c) 8(d) 4.2 9 (13)
906 Fig. 9 9 Differentials of edge and line. 10 1 Fig. 10 Extraction and deletion of lines (1). E x(x, y) = g x(x + i, y) g x(x + i, y) i r i r E y(x, y) = g y(x, y + j) g y(x, y + j) j r j r (13) r w 2w +1 { g if (E x + E y) th g line = 0 else (14) g notl = g g line g line g notl 10 11 (13) r 7 th 80 10 (c) 10 (b) 11 (c) 11 11 2 Fig. 11 Extraction and deletion of lines (2). (13) (14) 4.3
907 12 Fig. 12 Brightness and contrast adjustment. { g wk if f th g bright = g wk ( ) 0 else (15) g dark = g wk g bright f g bright g dark 12 (15) th 30 0.8 1 3 12 (b) Photoshop 12 (c) f 0 f 0 0 13 Fig. 13 Optimization of coefficients. 2 α j A i f t f j f 0 f 1 f 2 f 3 ( 3 min α j,β f t α jf j β) (16) i (x,y) A i j=0 f 0 β 13 13 (a) 3 20 30 RGB3 α 0 =0.81 α 1 =0.89 α 2 =1.73 α 3 =3.12 β =40 40 f 0 f 1 3.12 12 (c) 13 (b) 13 (b) GIMP Retinex Filter 14) 13 (c) 240 3 1.2 5.2
908 5. 5.1 1 PC DELL XPS600 PentiumD 820 2.80 GHz JAVA 4(a) 10 (a) 512 512 1 Poisson f 0 f st f wk 3 3 9 Poisson 0.1 5.2 1) Texture flattening 13) 11 (c) 10 Retinex Filter GIMP MSR Multi-Scale Retinex 15) 13 (c) 13 (b) Poisson Poisson 12) GPU MVC Mean-Value Coordinates 10) MVC 1 3 5.3 4.1 14 (a) w 14 (b) 14 (c) 1 Table 1 Computational time. Fig. 14 14 Comparison of gradient division methods.
909 6. Poisson 1) Perez, P., Gangnet, M. and Blake, A.: Poisson Image Editing, ACM Trans. Graphics (SIGGRAPH 03 ), Vol.22, No.3, pp.313 318 (2003). 2) Jia, J., Sun, J., Tang, C.-K. and Shum, H.-Y.: Drag-and-Drop Pasting, ACM Trans. Graphics (SIGGRAPH 06 ), Vol.25, No.3, pp.631 636 (2006). 3) D Vol.J90-D, No.7, pp.1686 1689 (2007). 4) 71 6R-2, pp.2-249 2-250 (2009). 5) Poisson Image Editing 25 NICOGRAPH (2009). 6) Fatal, R., Lischinski, D. and Werman, M.: Gradient Domain High Dynamic Range Compression, ACM Trans. Graphics (SIGGRAPH 02 ), Vo.21, No.3, pp.249 256 (2002). 7) Agrawal, A. and Raskar, R.: Gradient Domain Manipulation Techniques in Vision and Graphics, ICCV2007 Course (2007). 8) Orzan, A., et al.: Diffusion Curves: A Vector Representation for Smooth-Shaded Images, ACM Trans. Graphics (SIGGRAPH 08 ), Vol.27, No.3, pp.92:1 92:8 (2008). 9) McCann, J. and Pollard, N.S.: Real-Time Gradient-Domain Painting, ACM Trans. Graphics (SIGGRAPH 08 ), Vol.27, No.3, pp.93:1 93:7 (2008). 10) Farbman, Z., et al.: Coordinates for Instant Image Cloning, ACM Trans. Graphics (SIGGRAPH 09 ), Vol.28, No.32, pp.67:1 67:9 (2009). 11) Press., W.H., et al. (1993). 12) Poisson Image Editing Vol.9, No.2, pp.58 65 (2010). 13) Subr, K., Soler, C. and Durand, F.: Edge-preserving Multiscale Image Decomposition based on Local Extrema, ACM SIGGRAPH Asia 09, pp.147:1 147:9 (2009). 14) GIMP Retinex. http://docs.gimp.org/2.6/c/plug-in-retinex.html 15) Retinex Vol.67, No.4, pp.410 416 (2004). ( 22 3 31 ) ( 22 11 5 )