3 1 Table 1 1 Feature classification of frames included in a comic magazine Type A Type B Type C Others 81.5% 10.3% 5.0% 3.2% Fig. 1 A co

1 2 3 3 1 Hough 0.9 0.7 0.9 A Study on Frame Corner Detection of Comic Image Daisuke Ishii, 1 Kei Kawamura, 2 Keiichiro Hoashi, 3 Yasuhiro Takishima 3 and Hiroshi Watanabe 1 In this paper, we propose and evaluate a method to accurately detect the position of the corners of frames which consist a digital image of Japanesestyle comics. Existing automatic frame extraction methods are not capable of detecting the corners of the frames accurately. In order to solve this problem, this research proposes a method which combines signal processing techniques to detect the accurately positon of frames. First, the blank which exists between each frame is detected by the modified filling algorithm, to obtain the rough shapes of the frames. Next, corner detection and Hough-transform are applied to the resulting image. Finally, the results of these procedures are combined to obtain the individual position of frame corners. Evaluation experiments on actual comic images prove that the proposed method succeeds to detect frame corners from 0.7 to 0.9 precision with 0.9 recall. 1. 1) 1 2) 4) 1 Graduate School of Global Information and Telecommunication Studies, Waseda University 2 KDDI KDDI CORPORATION 3 KDDI KDDI R&D Laboratories 1 c 2010 Information Processing Society of Japan

3 1 Table 1 1 Feature classification of frames included in a comic magazine Type A Type B Type C Others 81.5% 10.3% 5.0% 3.2% 2 2. 2.1 1 5 Fig. 1 A comic image containing 5 frames. 2 Fig. 2 5 4 1 Frame reading order of the comic image shown in Fig.1 2 5) 8) 2 3 4 5 1 1 10 1 A B C 80% 2.2 2 5),6) 9) 2 2 c 2010 Information Processing Society of Japan

Start Step 1 Step 2 Step 3 Step 4 End Fig. 3 3 2 Steps of frame separation by the iterative separation method. At each step, the area surrounded by dotted lines indicates the target region and thick line shows the boundary for separation ρ = x cos θ + y sin θ (1) ρ θ 2 2 3 2 2 7),8) 2 3. 3.1 4 3.2 3 c 2010 Information Processing Society of Japan

Input Image Blank detection (fig.6) Line detection (Hough transform) (fig.8) Corner detection (Harris) (fig.7) Cross point detection on each Hough line (fig.9) 4 Frame s corner candidate detection (fig.10) Output corner(s) [6 10] Fig. 4 A flow of proposed method. Result of each step are shown in fig.[5 10] respectively. 20 5 6 3.3 Hough AND OpenCV Harris 250 5 Fig. 5 Start points of filling on blank detection. 6 Fig. 6 Result of blank detection. White resion means Blank. Most frames are filled in this step. Hough Hough OpenCV 100 30 2 AND 7 Hough 8,9 Hough 9 refintegration 4. 4.1 Precision 4 c 2010 Information Processing Society of Japan

7 Fig. 7 A result of corner detection on blank detected image. The result corners are mapped by circles 8 2 Table 2 Image size and number of page. Works Image size Page Comics A 740 1200 20 Comics B 760 1200 30 Comics C 1024 1536 30 Comics D 840 1200 30 Hough Hough Fig. 8 Result of Hough transform. Each result line is drown by lines. Many lines are along the frame border. Recall correctcorners obtainedcorners Precision =, (2) obtainedcorners correctcorners obtainedcorners Recall =. (3) correctcorners 6),8) 2 2 9 Fig. 9 The intersecting points of result lines of Hough transform. The intersecting points are plotted by circles. 10 Hough (AND) Fig. 10 Corner detection results of proposed method. These corners are obtained by integration (AND) of corner detection points(fig.7) and intersecting points of Hough transform result(fig.9.) 2 B C 4.2 11 ( )?? 110 3 4 Propose* Propose* Hough AND Propose Propose* Propose Hough Recall Precision 5 c 2010 Information Processing Society of Japan

:Result point of Proposed method :Border line of frame separation (conventional method) :Corner point of frame separation (conventional method) 4 Propose Propose* Table 4 Recall of the experiments on frame corner analysis. Recall of Propose is equivalent to Propose*. Works Propose Propose* Conv. I Conv. T Comics A 0.936 0.990 0.263 0.245 Comics B 0.958 0.972 0.179 0.196 Comics C 0.908 0.916 0.038 0.040 Comics D 0.976 0.984 0.267 0.202 Hough AND Recall Precision 11 Fig. 11 Corner detection results and border line of each method. 3 Propose* ( 7 Propose Table 3 Precision of the experiments on frame corner analysis. Propose* espresses the result of only corner detection (same as fig.7.) Propose has achieved highest precision. 5. Works Propose Propose* Conv. I Conv. T Comics A 0.936 0.588 0.260 0.258 Comics B 0.778 0.350 0.199 0.214 Comics C 0.708 0.159 0.043 0.057 Comics D 0.937 0.656 0.271 0.212 Hough AND 1) R&D 2009 2) D, vol.j74 D II, no.4, pp.491 499, April 1991. 3) K.Y. Wong, R.G. Casey, and F.M. Wahl, Document Analysis Systems, IBM J. Res. Develop., Vol.26, No.6, pp.647 656, Nov. 1982. 4) K. Kise, A. Sato, and M. Iwata, Segmentation of page images using the area Voronoi diagram, Computer Vision and Image Understanding, Vol.70, No.3, pp.370 382, 1998. 5) 2006, D 12 89, March 2006. 6) T. Tanaka, K. Shoji, F. Toyama, and J. Miyamichi, Layout Analysis of Tree- Structured Scene Frames in Comic Images, 20th International Joint Conference on Artificial Intelligence (IJCAI- 07), pp.2885 2890, Hyderabad, India, Jan. 6 12, 2007. 7) D, Vol.j90 D, No.7, pp.1667 1670, July 2007. 8) PRMU2009-34, pp.187 192, May 2009. 9) U. Ramer, An iterative procedure for the polygonal approximation of plane curves, Computer Graphics and Image Processing, 1, pp.244 256, Apr. 1972. 6 c 2010 Information Processing Society of Japan