Subspace 3 3 Projective Grid Space(PGS) 3 PGS % 10 Subspace 3 3 RANSAC

Size: px

Start display at page:

Download "Subspace 3 3 Projective Grid Space(PGS) 3 PGS % 10 Subspace 3 3 RANSAC"

ゆきかやぬま
5 years ago
Views:

1 2006 3

2 Subspace 3 3 Projective Grid Space(PGS) 3 PGS % 10 Subspace 3 3 RANSAC

3 Projective Grid Space Homography i

4 Subspace Subspace Subspace RANSAC A 73 A A A A A A ii

5 A A A A A A A B 83 iii

7 Projective Grid Space Projective Grid Space Homography f Base View ( ) ( ) m k (x) ( ) ( ) ( + ) ( ) ( ) v

8 5.8 3 ( + ) ( ) ( ) ( + ) ( ) ( ) ( + ) ( ) ( ) ( + ) ( ) ( ) ( + ) ( ) ( ) vi

9 4.1 [%] ( ) ( ) ( + ) ( ) ( ) ( + ) ( ) ( ) ( + ) vii

11 1 Graphical User Interface(GUI) 2 3 GUI Perceptual User Interface(PUI)[16] PUI 3 PUI Virtual Reality(VR) [7] [22, 15]

12 3 3 3 [30, 25, 13] 2 [20] 3 PUI 3 [20] PUI Subspace 3 3 Projective Grid Space[21] 3 3 Projective Grid Space Subspace M N 3M N Subspace Tomasi Shape from motion Subspace [6] Tomasi 2

13 Irani Subspace [18] Irani 3 3 RANSAC Subspace A 3

15 f (Perspective projection) (Focal point) (Focus) (Focal length) 3 x = f X Z (2.1) y = f Y Z 5

16 2.1: 3 (x, y, f) (2.1) x f y 0 f X Y Z 1 (2.2) 1 m = [x, y, 1] T, M = [X, Y, Z, 1] T (2.2) m P M (2.3) P = f f

17 (world coordinate system) (2.2) R t R RR T = R T R = I det(r) = 1 M c M w M c = RM w + t (2.4) M c = D M w (2.5) D [ R t D = 0 T 3 1 ], 0 3 = [0, 0, 0] T 3 (2.3) (2.5) m P M c = P D M w (2.6) (i) (ii) (iii) (c, x, y) c x y (o, u, v) x u u v (c, x, y) k u, k v u v 7

18 2.2: θ θ (o, u, v) [u 0, v 0 ] T (o, u, v) (c, x, y) m p = [u, v] T, m s = [x, y] T m p = A m s (2.7) A = k u k u cot θ u 0 0 k v / sin θ v A D A P = AP N D (2.8) f P N = 0 f 0 0 (2.9) M m m P M (2.10) 8

19 P H : 2 C 1 C 1 n C 1 C 1 d M C 1 M 1 n T M 1 = d (2.11) C 1 C 2 R t M C 2 M 2 M 2 = RM 1 + t n T M 1 /d = 1 M 2 = RM 1 + t nt M 1 (2.12) ] d = [R + tnt M 1 (2.13) d 9

20 M m 1 = M 1 /z 1 m 2 = M 2 /z 2 z 2 z 1 m 2 = [R + tnt d ] m 1 (2.14) (u 1, v 1 ) (u 2, v 2 ) α u 2 v 2 1 = H u 1 v 1 1 (2.15) (2.15) n 1 i (u 1i, v 1i ) 2 i (u 2i, v 2i ) (2.15) α α i α i u 2i v 2i 1 = H u 1i v 1i 1 (2.16) H (ij) H ij h = (H 11 H 12 H 13 H 21 H 22 H 23 H 31 H 32 H 33 ) T u 1i v 1i u 2i 0 0 u 1i v 1i v 2i 0 0 u 1i v 1i 1 1 [ h α i ] = (2.17) 3 9 U i 1 -u i n h U 1 u U 2 0 u h α 1. = (0) (2.18) U n 0 0 u 2n α n 10

21 h H (2.18) u 11 H 11 + v 11 H 12 + H 13 u 21 α 1 = 0 u 11 H 21 + v 11 H 22 + H 23 v 21 α 1 = 0 u 11 H 31 + v 11 H 32 + H 33 α 1 = 0 u 12 H 11 + v 12 H 12 + H 13 u 22 α 2 = 0 u 12 H 21 + v 12 H 22 + H 23 v 22 α 2 = 0 (2.19) u 12 H 31 + v 12 H 32 + H 33 α 2 = 0. u 1i H 11 + v 1i H 12 + H 13 u 2i α i = 0 u 1i H 21 + v 1i H 22 + H 23 v 2i α i = 0 u 1i H 31 + v 1i H 32 + H 33 α i = 0 α i = u 1i H 31 + v 1i H 32 + H 33 (2.19) α u 11 H 11 + v 11 H 12 + H 13 u 21 u 11 H 31 u 21 v 11 H 32 u 21 H 33 = 0 u 11 H 21 + v 11 H 22 + H 23 v 21 u 11 H 31 v 21 v 11 H 32 v 21 H 33 = 0 u 12 H 11 + v 12 H 12 + H 13 u 22 u 12 H 31 u 22 v 12 H 32 u 22 H 33 = 0 u 12 H 21 + v 12 H 22 + H 23 v 22 u 12 H 31 v 22 v 12 H 32 v 22 H 33 = 0 (2.20). u 1i H 11 + v 1i H 12 + H 13 u 2i u 1i H 31 u 2i v 1i H 32 u 2i H 33 = 0 u 1i H 21 + v 1i H 22 + H 23 v 2i u 1i H 31 v 2i v 1i H 32 v 2i H 33 = 0 A h = 0 u 11 v u 21 u 11 u 21 v 11 u u 11 v 11 1 v 21 u 11 v 21 v 11 v 21 u 12 v u 22 u 12 u 22 v 12 u u 12 v 12 1 v 22 u 12 v 22 v 12 v u 1i v 1i u 2i u 1i u 2i v 1i u 2i u 1i v 1i 1 v 2i u 1i v 2i v 1i v 2i H 11 H 12 H 13 H 21 H 22 H 23 H 31 H 32 = (0) (2.21) H 33 (2.22) h A T A 11

22 : 2 (Epipolar plane) (Epipolar line) (Epipole) 1 1 m 2 m R t x T (t (R x T + t)) = 0 (2.23) x 1 x 2 x 3 = 0 x 3 x2 x 3 0 x 1 x 2 x 1 0 (2.24) x T [t] (R x + t) = x T E x = 0 (2.25) 12

23 E = [t] R E (Essential Matrix) x T E x = m T F m = 0 (2.26) F = A T EA 1 (2.27) F (Fundamental Matrix) e e M 2 m m (2.28) m T F m = 0 (2.28) 1 m l l = F m (2.29) l (2.28) m T l = 0 2 m 2 l 1 m 2 l l = F T m l 2 m m (2.28) F m = 0 m e F ẽ = 0 (2.30) e 2 m 1 1 F T ẽ = 0 e 2 13

24 3 3 3 F = f 11 f 12 f 13 f 21 f 22 f 23 = f 1 f 2 (2.31) f 31 f 32 f 33 f 3 rankf = 2 3 f 1 f 2 f (2.30) 3 0 f 1 ẽ = 0 f 2 ẽ = 0 f 3 ẽ = 0 (2.32) e f 1 f 2 f 3 F 3 f 1 f 2 f f 1 f 2 f 3 e F 1 e F ( ) ( ) 2 e F ( ) ( ) 2.5: l l m m m = u 1 0 m = 14 u 1 0 (2.33)

25 m m m T F m = 0 (2.34) f 11 uu + f 12 u + f 21 u + f 22 = 0 (2.35) [ ] λ u 1 = [ f 21 f 22 f 11 f 12 ] [ u 1 ] (2.36) H [ H = f 21 f 22 f 11 f 12 ] (2.37) H F 2 e e 2 3 H [ ] RH RHe F = (2.38) e T RH e T RHe R = [ ] 2 2 = [10, 11, 14, 27] 8 1 m i = [u i, v i, 1] T 2 m i = [u i, v i, 1] T (2.39) m T i F m i = 0 (2.39) 15

26 F (2.39) u T i f = 0 (2.40) u i = [u i u i, u i v i, u i, v i u i, v i v i, v i, u i, v i, 1] T f = [F 11, F 12, F 13, F 21, F 22, F 23, F 31, F 32, F 33 ] T F ij i j n (2.40) Bf = 0 B B = u T 1. u T n min F ( m T i F m i) 2 (2.41) i f min f f T B T Bf (2.42) f = 1 (2.42) f B T B 2 2 F F = UΣV T (2.43) 16

27 Σ σ 1 > σ 2 > σ 3 Σ = diag(σ 1, σ 2, σ 3 ) U V F 2 F = U ˆΣV T (2.44) ˆΣ = diag(σ 1, σ 2, 0) 2.4 ( ) Projective Grid Space Homography F 3 [36, 27, 29] P P P = [I 0] (2.45) P = [[ẽ ] F ẽ ] (2.46) e Projective Grid Space 3 [21] 2 (Base View) 2.6 Grid Point Base View 1 (p, q) 1 Base View 2 r s Base View 1 Base 17

28 View 2 F 21 l Base View 2 l Projective Grid Space Projective Grid 2.6: Projective Grid Space Space 2 (p, q, r) (p, q, r) (p, q) (r, s) s s = (l xr + l z ) l y (2.47) l = (l x, l y, l z ) (p, q) Base View 2 l = F 21 p q 1 (2.48) Projective Grid Space i 2 F i1 F i2 (p, q, r) i (p, q, r) Base View 1 (p, q) i F i1 (p, q) l 1 Base View 2 (s, r) i Base View2 (s, r) F i2 l i l 1 l 2 (p, q, r) 18

29 2.7: Projective Grid Space P P = p 11 p 12 p 13 p 14 p 21 p 22 p 23 p 24 p 31 p 32 p 33 p 34 [u, v] u = p 11X w + p 12 Y w p 13 Z w + p 14 p 31 X w + p 32 Y w p 33 Z w + p 34 v = p 21X w + p 22 Y w p 23 Z w + p 24 p 31 X w + p 32 Y w p 33 Z w + p 34 P P p 34 1 X w p 11 + Y w p 12 + Z w p 13 + p 14 ux w p 31 uy w p 32 uz w p 33 = u X w p 21 + Y w p 22 + Z w p 23 + p 14 vx w p 31 vy w p 32 vz w p 33 = v [X w, Y w, Z w ] [u, v] P

30 6 P Tsai [4, 5] Homography [24, 31] 2.8 0(Z = 0) (2.3) 2.8: Homography x y 1 P X Y 0 1 p 11 p 12 p 14 p 21 p 22 p 24 p 31 p 32 p 34 X Y 1 ˆP X Y 1 H X Y 1 (2.49) P ˆP 2 (Homography) Homography Homography 20

31 Homography A = f 0 c x 0 f c y (c x, c y ) : f : (2.50) A 1 H = [r 1 r 2 t] (2.51) R r 1 r 2 0 f 2 = (h 11 c x h 31 )(h 12 c x h 32 ) + (h 21 c y h 31 )(h 22 c y h 32 ) h 31 h 31 (2.52) R, t (2.51) r 1, r 2, t r 3 r 1 r 2 r 3 R R = [r 1 r 2 (r 1 r 2 )] (2.53) Homography 21

33 ( ) f(x, y, z; t) = P i I i = I i (u i, v i ; t) x = (x, y, z) u = (u i, v i ) u i = [P i] 1 (x, y, z, 1) T [P i ] 3 (x, y, z, 1) T (3.1) v i = [P i] 2 (x, y, z, 1) T [P i ] 3 (x, y, z, 1) T (3.2) [P i ] j P i j x u i 2 3 u i x u i x u i u i x (3.3) x 23

34 3.1: f 24

35 u x i f = f u i x u i x x u i = ( ) (3.4) x = f x = (0 0) (3.5) u i u i t x m E E = E(m; x; t) (x, y, z) t s = s(x, y, z; t) s(x, y, z; t) = E(m; x, y, z; t)dm (3.6) S(n) S(n) = {m : m = 1 and m n 0} ρ = ρ(x; t) x = (x, y, z) i u i I(u i ; t) = C ρ(x; t)[n(x; t) s(x; t)] (3.7) C x(t) i u i (t) 3 dx dt du i ( dt ) x dρ dt = 0 (3.8) di i dt = I i du i dt + I i t = C ρ(x; t) d [n s] (3.9) dt 25

36 I i du i I i dt t n s n s n s = S(n) E(m; x; t)n dm (3.10) d [n s] = 0 dt x i u i u i du i dt = u i dx x dt (3.11) f dx dt = x dx u i dt + x (3.12) t ui 3 2 x u i u i x x t f ui 3.3 [2] 26

37 3.3.1 ( ) 3.2 A a B a (N N) B b (a ) ( A) 3.2: ( B) I N N T,. (x, y). (3.13) (x, y),. R(x, y) = N 1 n=0 N 1 m=0 I(x + m, y + n) T (m, n) (3.13), ( ) (x, y) t E(x, y, t) t ( x, y) (x, y) (x + x, y + y) 27

38 E(x, y, t) = E(x + x, y + y, t + t) (3.14) E(x, y, t) =.. E(x, y, t) + x E x E + y y + t E t (3.15) t t 0 x E t x + y E t y + E t = 0 (3.16) x E t x + y E t y + E t = 0 (3.17) (x, y) x x y = u y = v t t E x, E y E t ( (3.18)) E x u + E y v + Et = 0 (3.18) u v 2 u v ( ( ) ) ( ( ) ) Lucas-Kanade (u, v) Lucas-Kanade (LK) Lucas-Kanade (u, v) [1] 28

39 3 3. E x1 u + E y1 v = E t1 E x2 u + E y2 v = E t2. (3.19) E x9 u + E y9 v = E t9 (u, v) E E = E x1 E x2. E y1 E y2. (3.20) E x9 E y9 E 9 2 t = (E t1, E t2,, E t9 ) T P = (u, v) T EP = t E E T EP = E T t (3.21) E T E 2 2 E T t 2 1 [ ] [ Ex E x Ex E y Ex E y Ey E y u v ] = [ ] Ex E t Ey E t (3.22) (3.23) (u, v) [ u v ] [ ] 1 [ ] Ex E x Ex E y Ex E t = Ex E y Ey E y Ey E t (3.23) 29

41 Projective Grid Space M M Base View 1 m 1 m 1 M Base View 2 m Base View 2 m 2 m 2 m 1 Base View 2 l : Base View Base View M 31

42 i Base View 1 m 1 Base View 1 i F i1 l i1 Base View 2 m 2 Base View 2 i F i2 l i2 4.2: Step1 t t 1 Step2 Projective Grid Space ( Base View1 ) Base View1 Base View2 Step3 C i Step4 2 Base View Step5 Step2 Step4 Seitz Voxel Coloring 32

43 1 Z [19, 28] 0 Z Z 1 Z camera 1 x y 45 camera 2 camera 3 camera 1 x z x 4.3: y (b) 33

44 4.4: 2 ( ) 34

45 : 2 ( ) Point Gray Research Dragonfly(IEEE-1394 ) 35

46 t 4.6: t 1 3 t t p q r Projective Grid Space q + ( ) 4.1: [%] p + p q + q r + r

47 4.7: 4.8: 3 37

49 5 Subspace 3 Projective Grid Space Subspace Vedula [20] Vedula 3 2 Vedula 3 (Voxel) du k dt = u k x dx dt (5.1) du k dt k u k dx dt x u k x x = [X x, X y, X y ] T 39

50 5.1: m k (x) k u k = [u k, v k ] T (5.1) dx dt = ( ) uk du k x dt + µr k(u k ) (5.2) ( ) uk x : u k x r k (u k ) : u k µ : m k (x) dx [( ) ] dt uk du k x dt r k(u k ) dx dt = 0 (5.3) (5.3) m k (x) 3 x 5.1 m k (x) m k (x) M(x) = k ˆm k ˆm T k (5.4) ˆm k m k (x) m k (x) (5.4) M(x) 0 M(x) Voxel Seitz Voxel Coloring [19] 40

51 5.1.2 (5.1) n u 1 t v 1 t. u n t v n t = u 1 X x u 1 X y u 1 X z v 1 X x v 1 X y v 1 X z u n X x v n X x. u n X y v n X y u n X z v n X z dx dt (5.5) dx 3 (5.6) 2n dt x Subspace j x i dx ij t dt j ω j ( 5.2 ) x dx ij dt v ij v ij = t j + ω j x i (5.6) v ij = [v xij, v yij, v zij ] T x i = [X xi, X yi, X zi ] T t j = [t xj, t yj, t zj ] T : ω j = [ω xj, ω yj, ω zj ] T : i : j : 41

52 5.2: (5.6) v xij v yij v zij = t xj ω zj ω yj 0 t yj 0 ω zj 0 ω xj 0 0 t zj ω yj ω xj X xi X yi (5.7) X zi (5.7) v xij v yij = s xj s yj q i(6 1) (5.8) v zij s zj (3 6) s xj = [t xj, 0, 0, 0, ω zj, ω yj ] s yj = [0, t yj, 0, ω zj, 0, ω xj ] s zj = [0, 0, t zj, ω yj, ω xj, 0] q i = [1, 1, 1, X xi, X yi, X zi ] T (5.7) (5.8) v xj v yj = s xj s yj Q (6 N) (5.9) v zj (3 N) s zj 42

53 Q (6 N) = [q 1,..., q N ] = X x1... X xn X y1... X yn X z1... X zn (5.9) s xj, s yj, s zj M (5.10) S x = s x1., S y = s y1., S z = s z1. s xm s ym s zm V x V y = S x S y Q (6 N) (5.10) V z (3M N) S z (3M 6) [V x /V y /V z ] Q S x S y S z 1 [S x /S y /S z ] 3 [V x /V y /V z ] Q [S x /S y /S z ] : [V x /V y /V z ] Q 1 Q

54 5.2.2 Subspace 2 3 Subspace Suspace v x v y v z V x V y = U 1 DU T 2 (5.11) V z (3M N) U 1 3M 3M U 2 N N D D d = [d 1, d 2,..., d 3M ] T 3 d = [d 1, d 2, d 3, 0,..., 0] T D d d D 0 0 d = (3M N) D U 1 U 2 V x V y V z (3M N) = U 1 D U T 2 (5.12) [V x/v y/v z] 3 (5.6)

55 Subspace 5 1 (M = 2) Subspace : 3 ( ) (a) Subspace (b) v t v (5.13) (5.14) : cos θ = v v t v v t (5.13) 45

56 5.4: 3 ( ) 5.5: 3 ( + ) 5.6: 3 ( ) 46

57 5.7: 3 ( ) 5.8: 3 ( + ) 5.9: 3 ( ) 47

58 5.10: 3 ( ) 5.11: 3 ( + ) 48

59 : v v t v t (5.14) (a) (b) 5.12: ( ) 5.13: ( ) % 94.2% 10% 46.3% 58.3% 49

60 5.14: ( + ) 5.15: ( ) 50

61 5.16: ( ) 5.17: ( + ) 51

62 5.18: ( ) 5.19: ( ) 52

63 5.20: ( + ) 5.21: ( ) 53

64 5.3.2 実画像を用いた実験実画像を用いたモーション復元実験を行う実験環境は図 5.22 に示すように 3 次元空間中に配置した 5 台のカメラを用い各カメラのパラメータは既知であるとする各カメラ間のシャッタータイミングは同期している対象とする物体は図 5.23 に示すようにルービックキューブを持ち空間内で運動する人間の腕部とするオプティカルフローの計算にはブロックマッチング法を用いる本実験では y 軸方向に 1 秒間に約 15cm 平行移動した場合と x 軸を中心に約 180 度回転した場合の 2 つのシーケンスを対象とし入力画像は基準フレームから前後 1 フレームの計 15 枚 (3 フレーム 5 台) を用いる図 5.23 に示す平行移動における入力オプティカルフローから復元されたシーンフローを図 5.24 に示すまた回転運動に対する復元されたシーンフローを図 5.25 に示す Vedula ら図 5.22: 実験環境の手法ではシーンフローの存在位置を各入力カメラから算出されるべクトル mk の一致性から判定するこれにより図 (a) ではシーン内の唯一の動物体である腕部のシーンフローのみを復元することが可能であることが分かる図 5.24(a) では分散しているシーンフローの方向が Subspace 拘束を用いることで同一の方向へ修正されているのが分かるまた図 5.25(b) でも同様に分散しているシーンフローの方向を修正できていることが分かる 54

65 図 5.23: 実画像における入力画像とオプティカルフロー (平行移動) 図 5.24: 平行移動における 3 次元シーンフローの例図 5.25: 平行移動における 3 次元シーンフローの例 55

67 x i v ij j (5.6) v ij = 0 X zi X yi X zi 0 X xi X yi X xi ] = [ [x i ] I (3 3) ] [ ω j t j [ ωj [x i ] x i, I (3 3) : 3 3 t j ] (6.1) N v 1j. v Nj = [x 1 ] I 1(3 3). [x N ]. I N(3 3) [ ωj t j ] (6.2) ω j = [ω xj, ω yj, ω zj ] T t j = [t xj, t yj, t zj ] T 6 (6.2) 3 N [x i ] 2 N = 2 (6.2) 3 v ij x i 57

68 6.1.1 RANSAC 5.2 Subspace 3 Subspace ( 6.1 ) 6.1: RANSAC [3] RANSAC Step1 3 Step2 2 Step3 Step4 Step5 Spte6 Step1 Step5 58

69 (a) 6.1: ( ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] 1.75E E E E E E-05 RANSAC E E E-05 RANSAC 7.15E E E E : ( ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] 5.48E E E E-06 RANSAC E RANSAC E E : ( + ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] 4.25E E E E E-05 RANSAC RANSAC E E-05 RANSAC RANSAC 59

70 6.4: ( ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] 4.43E E E E E-05 RANSAC E RANSAC 5.89E E : ( ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] 7.61E E E E E-05 RANSAC E E E-05 RANSAC 8.77E E E : ( + ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] E E E E-05 RANSAC 1.68E E RANSAC E : ( ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] 4.23E E E E E E-06 RANSAC E-05 RANSAC 2.31E E E-05 60

71 6.8: ( ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] E E E-06 RANSAC E E-05 RANSAC E E E E : ( + ) ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] E E E-06 RANSAC E-05 RANSAC E : 61

72 6.2(b)(c) RANSAC Subspace RANSAC : ω x ω y ω z t x t y t z [deg/frame] [deg/frame] [deg/frame] [mm/frame] [mm/frame] [mm/frame] ω x, ω y, ω z 1 y 1frame(1/30sec) 4.5mm cm 15cm x 1frame frame 6 t y t y 62

73 Subspace Projective Grid Space 3 3 Projective Grid Space 5 Subspace 3 Subspace RANSAC 63

74 2 ( ) 64

75 65

77 [1] B. D. Lucas, and T. Kanade, An iterative image registration technique with an application to stereo vision, Imaging Understanding Workshop, pp , [2] BKP Horn, BG Schunck, Determining optical flow, Artificial Intelligence, 17, pp , [3] M. Fischer, and R. Bolles. Randoam sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM, Vol. 24, No. 6, pp , [4] R. Y. Tsai, An efficient and accurate camera calibration technique for 3D machine vision, CVPR, pp , [5] R. Y. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-self TV camera and lenses, J-RA, vol. 3, pp , [6] C. Tomasi and T. Kanade, Shape and Motion from Image Streams under Orthography: a Factorization Method, International Journal of Computer Vision, Vol. 9, No. 2, pp , [7] T. Baudel and M. B. Lafon, CHARADE : Remote Control of Objects Using Free- Hand Gestures, Communication of the ACM, Vol. 36, No. 2, pp , [8] Q.T.Luong, T.Vieville, Canonic representations for the geometries of multiple projective views, Proc. 3rd Europian Conference on Computer Vision, Vol.1: , 1994 [9] J. L. Barron, D. J. Fleet, and S. S. Beauchemin, Performance of Optical Flow Techniques, International Journal of Computer Vision, Vol. 12, No. 1, pp , [10],,, Vol.35, No.2, pp ,

78 [11],,, Vol. 36, No. 8, pp , [12] R. T. Collins, A space-sweep approach to true multi-image matching, In IEEE Computer Vision and Pattern Recognition, pp , [13],,, 3,. D-II,, II-, Vol.79, No.8, pp , [14],,, Vol. 37, No. 3, pp , [15],,,,,,, Vol.51, No.12, pp , [16] M. Turk, Moving from GUIs to PUIs, Proc. Fourth Symposium on Intelligent Information Media, Tokyo, Japan, [17] S. M. Seitz and K. N. kutulakos, Plenoptic Image Editing, Proc. 6th Int. Conf. Computer Vision, [18] M. Irani, Multi-Frame Optical Flow Estimation Using Subspace Constraints, International Conference on Computer Vision, Vol. 1, pp [19] S. M. Seitz and C.R. Dyer, Photorealistic scene reconstruction by Vvoxel Coloring, International Journal of Computer Vision, vol. 35, No. 2, pp , [20] S. Vedula, S. Baker, P. Rander, R. Collins, T. Kanade, Three-Dimensional Scene Flow, Proceedings of the 7th International Conference on Computer Vision, Vol. 2, September, pp , [21] H. Saito, T. Kanade, Shape Reconstruction in Projective Grid Space from Large Number of Images, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 99), June, [22],,,,, Vol.40, No.2, pp , [23] J.Y. Bouguet, Pyramidal Implementation of the Lucas Kanade Feature Tracker, In OpenCV Documentation, Microprocessor Research Labs, Intel Corp.,

79 [24] G. Simon, and A. Ritzgibbon, and A. Zisserman, Markerless Tracking using Planar Structures in the Scene, Proc. of the ISAR, pp , [25] Makoto Hirose, Takeo Miyasaka,Kazuhiro Kuroda and Kazuo Araki, Integration of successive range images for robot vision, Proc. of XIXth Congress of the international society for photogramm etry and remote sensing(isprs), pp , Amsterdam, July [26],, Vol.42(SIG6):33-43, [27],,, J85-D-2(3): , [28] S. M. Seitz and K. N. Kkutulakos, Plenoptic Image Editing, International Conference on Computer Vision, vol. 48, No. 2, pp , [29],,, (MIRU2004) [30],, 3D, 2005 pp , [31],,, (MIRU2005) pp , [32] - - [33] [34] 3 CG [35] G [36] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press,

81 [1],,. 3, (MIRU2006), p , [2],,. 3, CVIM 149, pp , [1],,. Subspace 3, 18, September, [2],,. Projective Grid Space 3,, 0-324,

83 A A.1 A.1.1 V a, b 2 (i) (ii) a + b V k ka ka V 8 V a, b a + b = b + a (k + l)a = ka + la (a + b) + c = a + (b + c) k(a + b) = ka + lb a + 0 = a (kl)a = k(la) a + ( a) = 0 1a = a 73

84 A.1.2 n a 1, a 2,..., a n c 1 a 1 + c 2 a c n a n = 0 (A.1) c 1 = c 2 =... = c n = 0 a 1, a 2,..., a n (A.1) c 1, c 2,..., c n ( 0 ) a 1, a 2,..., a n a 1, a 2,..., a n A = [a 1, a 2,..., a n ] n a 1, a 2,..., a n n A n a 1, a 2,..., a n v = v 1 a 1 + v 2 a v n a n (A.2) v V u = u 1 a 1 + u 2 a u n a n V (A.3) v = v 1 a 1 + v 2 a v n a n V (A.4) u + v = (u 1 + v 1 )a 1 + (u 2 + v 2 )a (u n + v n )a n V (A.5) ku = ku 1 a 1 + ku 2 a ku n a n V (A.6) V a 1, a 2,..., a n V V n V dim V 3 x, y, z 3 [v x, v y, v z ] T v x v y v z 1 = v x v y v z 0 0 1

85 e x = [1, 0, 0] T, e y = [0, 1, 0] T, e z = [0, 0, 1] T A.1.3 n R n 2 a = [a 1, a 2,..., a n ] T, a = [b 1, b 2,..., b n ] T (a, b) (a, b) = n a i b i = a 1 b 1 + a 2 b a n b n i=1 (A.7) (1) (a, b) = (b, a) (2) (a, b + c) = (a, b) + (a, c) (3) (ka, b) = k(a, b) (4) (a, a) 0 n a, b n e 1, e 2,..., e n a = a 1 e 1 + a 2 e a n e n, b = b 1 e 1 + b 2 e b n e n (e i, e j ) = δ ij = { 1 (i = j) 0 (i j) (A.8) (a, b) (a, b) = (a 1 e 1 + a 2 e a n e n, b 1 e 1 + b 2 e b n e n ) = a 1 b 1 (e 1, e 2 ) + a 2 b 2 (e 1, e 2 ) a 2 b 1 (e 2, e 1 ) +a 2 b 2 (e 2, e 2 ) a n b n (e 2, e 1 ) = a 1 b 1 + a 2 b a n b n R n 75

86 A.2 A.2.1 R n n e 1, e 2,..., e n R n x x = n x i e i i=1 R m y m ẽ 1, ẽ 2,..., ẽ n y = m x i ẽ i i=1 2 y x y 1 = a 11 x 1 + a 12 x a 1n x n y 2 = a 21 x 2 + a 22 x a 2n x n. y m = a m1 x 2 + a m2 x a mn x n a ij (i = 1, 2..., m, j = 1, 2,..., n) m n y 1 a 11 a a 1n x 1 y 2. = a 21 a a 2n x 2... y m a m1 a m2... a mn x n A x x 1, x 2,..., x n A(x + x ) = Ax + Ax A(cx) = c(ax) 76

87 c f A.2.2 f : x y(x R n, y R m ) R n e 1, e 2,..., e n e 1, e 2,..., e n p 11 p p 1n p [e 1, e 2,..., e n ] = [e 1, e 2,..., e n ] 21 p p 2n (A.9).. p n1 p n2... p nn P P e i(i = 1, 2,..., n) f R m (A.9) e i = n e k p ki f k=1 ( n ) f(e i) = f e k p ki = k=1 n a jk p ki = (AP ) ji k=1 n f(e k )p ki = k=1 m n a jk p ki ẽ j j=1 k=1 f(e i) = m (AP ) ji ẽ j j=1 (A.10) f AP R m ẽ 1, ẽ 2,... ẽ n ẽ 1, ẽ 2,... ẽ n [ẽ 1, ẽ 2,... ẽ n] = [ẽ 1, ẽ 2,... ẽ n ]Q Q m m [ẽ 1, ẽ 2,... ẽ n ] = m [ẽ 1, ẽ 2,... ẽ n]q 1 (Q 1 ) lj ẽ j = ẽ l(q 1 ) lj (A.10) l=1 f(e i) = m m (AP ) j i ẽ l(q 1 ) lj = j=1 l=1 77 m m (Q 1 ) lj (AP ) ji ẽ l l=1 j=1

88 m j=1 (Q 1 ) lj (AP ) ji = (Q 1 AP ) li f(e i) = m (Q 1 AP ) li ẽ l (A.11) l=1 f A = Q 1 AP A.2.3 R n W W R n (1) a W, b W a + b W (2) a W ca W (c ) 3 A.2.4 f : R n R m R n x f(x) = 0 R m Kerf f(x) R m R n x f f(x) Imf Kerf = {x R n f(x) = 0 R m } Imf = {f(x R m x R n } Kerf R n Imf f(e 1 ), f(e 2 ),... f(e n ) R m R n n f : R n R m dim(kerf) + dim(imf) = n dim(kerf) dim(imf) 78

89 A.2.5 Imf f(x) ( n ) f(x) = f x i e i = i=1 n x i f(e i ) i=1 (A.12) Imf f(e i )(i = 1, 2,..., n) R m f(e i )(i = 1, 2,..., n) Imf f A f(e i ) A = [f(e 1 ), f(e 2 ),..., f(e n )] A f(e 1 ), f(e 2 ),..., f(e n ) R n R m f rankf = ranka A.3 A.3.1 A P P 1 AP 2 2 [ ] [ ] P 1 a 11 a 12 λ 1 0 P = a 21 a 22 0 λ 2 P P = [x 1 x 2 ] P P 1 x 1 x 2 c 1 x 1 +c 2 x 2 = 0 c 1 = c 2 = 0 (A.13) [ ] [ ] a 11 a 12 λ 1 0 [x 1 x 2 ] = [x 1 x 2 ] a 21 a 22 0 λ 2 = [λ 1 x 1 λ 2 x 2 ] 79

90 x 1 x 2 [ ] [ ] a 11 a 12 x 1 = λ 1 x 1, a 11 a 12 x 2 = λ 2 x 2 a 21 a 22 a 21 a 22 λ 1 λ 2 A x 1, x 2 A n n A R n [e 1 e 2... e n] = [e 1 e 2... e n ]P A = P 1 AP A A 3 A T A (A T A) T = A T A A T A A T A T 1 T 1 1 A T AT 1 = γ γ γ 3 γ 1, γ 2, γ 3 A T A u i γ i T 1 T 1 = [u 1, u 2, u 3 ] (Au i, Au i ) = (u i, A T Au i ) = γ i (u i, u i ) = γ i (Au i, Au i ) 0 γ i 0 γ 1 = λ 2 1 0, γ 2 = λ 2 2 0, γ 3 = λ v 1 = Au 1 λ 1, v 2 = Au 2 λ 2 v 1 v 2 80

91 v 3 2 T 2 = [v 1, v 2, v 3 ] (v i, v j ) = δ ij T T T = I(I = ) T 2 T T 2 AT 1 = (v 1, Au 1 ) (v 1, Au 2 ) (v 1, Au 3 ) (v 2, Au 1 ) (v 2, Au 2 ) (v 2, Au 3 ) (v 3, Au 1 ) (v 3, Au 2 ) (v 3, Au 3 ) = λ λ (A.13) i = 1, 2 j = 1, 2, 3 (v i, Au j ) = (v 3, Au j ) = 0 ( ) Aui, Au j = 1 λ (u i, A T Au j ) = λ j δ i j λ i n A 2 T 1, T 2 λ λ T T 2 AT 1 = λ n λ 2 1, λ 2 2,..., λ 2 n AT A λ 1, λ 2,..., λ n A 81

93 B [1],,. 3, (MIRU2006), p , [2],,. 3, CVIM 149, pp , [3],,. Subspace 3, 18, September, [4],,. Projective Grid Space 3,, 0-324,

Optical Flow t t + δt 1 Motion Field 3 3 1) 2) 3) Lucas-Kanade 4) 1 t (x, y) I(x, y, t)

http://wwwieice-hbkborg/ 2 2 4 2 -- 2 4 2010 9 3 3 4-1 Lucas-Kanade 4-2 Mean Shift 3 4-3 2 c 2013 1/(18) http://wwwieice-hbkborg/ 2 2 4 2 -- 2 -- 4 4--1 2010 9 4--1--1 Optical Flow t t + δt 1 Motion Field