Control of Data Flows Based on Drag-and-Drop between Appliances Using a Camera Phone Satoru MITSUI Thesis submitted in partial fullfillment of the req

Size: px

Start display at page:

Download "Control of Data Flows Based on Drag-and-Drop between Appliances Using a Camera Phone Satoru MITSUI Thesis submitted in partial fullfillment of the req"

しなつおおはし
5 years ago
Views:

1 U

2 Control of Data Flows Based on Drag-and-Drop between Appliances Using a Camera Phone Satoru MITSUI Thesis submitted in partial fullfillment of the requirements for the degree of Master in Information and Computer Science Student ID 3606U104-2 Submission Date February 4, 2008 Supervisor Professor Tatsuo Nakajima Department of Computer Science Graduate School of Science and Engineering Waseda University

3 iii

4 iv Abstract In this thesis, I propose an interaction model to be able to control the connection between information appliances in the next-generation home network environment by using a camera phone. More information appliances which have the remote accessibility there are, more connections between information appliances are expected. This system adopts the extended direct manipulation technique based on drag-and-drop model in graphic user interface and allows users realize data flows and those directions between information appliances. By using this system, users are not confused about appliances manipulations even in the environment which has many multifunctional information appliances. The evaluation experiment showed that the proposed interaction model provide the usability and the ease of learning to users.

5 i Abstract iii iv iv v Pick-and-drop mediablocks InfoStick Touch-and-connect transtick DataTiles GUI

6 ii UI

7 iii 1 1 [13] [11] GUI Sony Type X X X UI

8 iv Nokia N VisualCodes

9 v 4 1 VisualCode ID UDN PC

10 1 1,,. 1.1 e-life [10] PDA PC DVD AV PC [2] 1 1 [13] 1 2 DLNA DLNA PC

11 : [13] LAN 1.2 [14] Sony

12 : [11]

13 ECHONET AV IEEE1394 DLNA 1 PC IP GUI 2.4 Shneiderman direct manipulation technique [8] Pick-and-drop[5] GUI 1 2 ECHONET

15 Pick-and-drop Pick-and-drop[5] ID mediablocks mediablocks[9] MIT Ullmer PC

16 2 7 ID InfoStick InfoStick[4] SFC CCD InfoStick ID InfoStick Touch-and-connect Touch-and-connect[3] transtick transtick[12]

17 2 8 PC DataTiles DataTiles[6] Ullmer 2.3 GUI GUI

18 Shneiderman Direct manipulation[8]

19 10 3 GUI IP IP AV AV DLNA UPnP PC ECHONET UPnP PC

20 : UPnP 3.1.2

21 GUI : GUI GUI

22 :

23 : 3 5:

24 GUI 3 6:

25 3 16 GUI 3 7 コンテンツ連携 ( 例 : ディスプレイの静止画をプリンタで印刷する ) ディスプレイスナップショットを撮る画像取得プリンタ印刷するデバイス選択 (Slect Device) 機能選択 (Select Function) データ選択 (Select Data) 移動デバイス選択 (Select Device) 機能選択 (Select Function) 移動元デバイスでの操作移動先デバイスでの操作連携印刷設定の UI 枚数濃さ両面? 印刷範囲 3 7: GUI Web

26 :

Function) データ選択 (Select Data) 移動デバイス選択 (Select Device) 機能選択

27 3 18 GUI コンフィグレーション連携 ( 例 : 温度計をもとにエアコンの温度を自動調整する ) 温度計温度を送るデータ取得エアコン温度で自動調整デバイス選択 (Slect Device) 機能選択 (Select Function) データ選択 (Select Data) 移動デバイス選択 (Select Device) 機能選択 (Select Function) 移動元デバイスでの操作移動先デバイスでの操作連携自動接続設定の UI 温度と室温の調整期間など 3 9: 2

28 DataFlowController DataFlowController UPnP

29 PC PC PC PC GUI AV Augmented Reality [1] CCD ID DataFlowController

30 DataFlowController , 4 4 X 4 5 X 4 6 X 4 7

31 :

32 : 2 4 3: 4 4:

33 UI : Sony Type X 4 6: X 4 7: X UI

34 第4章システムの設計と実装 25 図 4 8: 再生可能な動画のリスト図 4 9: 移動元デバイスを選択した状態図 4 10: ディスプレイの機能図 4 11: 動画再生中のディスプレイ図 4 12: 動画再生中のコントロール UI

35 第4章システムの設計と実装 26 図 4 13: 時計デバイスのアイコン図 4 14: 時計デバイスの機能移動元選択時図 4 15: 時計デバイスの時刻データダイ図 4 16: 時計デバイスの機能移動先選アログ択時図 4 17: 時計の同期

36 : X DataFlowController : 1

37 PC Nokia N93 S60 3rd 4 20 N93 LAN UPnP Symbian C++ TCP/IP API Symbian C++ Sony Type X 4 5 PC Java J2SE UPnP SOAP 1. VisualCodes 2. CyberLink UPnP UI

4 29 4 20: Nokia N93 VisualCodes VisualCodes[7]

38 : Nokia N93 VisualCodes VisualCodes[7] ID Symbian S60 2nd API 3rd ID 4 21: VisualCodes

39 4 30 CyberLink CyberLink UPnP UPnP CyberLink C++ Java Perl C C Symbian C++ 30,000 XML CyberLink LAN SOAP QuickTime for Java Sony Type X MPEG-2 (MPEG-2 PS) MPEG-2 PC API QuickTime for Java CyberLink ID ApplianceInfo 16 description.xml UDN D&D D&D UI UPnP UI CyberLink:

4 31 Destination Device Source Device Service Application

Visual Tag Manipulation Destination Service UPnP Controller

Controller Source Service UI Controller discovery Camera

40 4 31 Destination Device Source Device Service Application Service Application UPnP UPnP Manipulation Visual Tag Visual Tag Manipulation Destination Service UPnP Controller Destination Service UI Controller UPnP Source Service UPnP Controller Source Service UI Controller discovery Camera Module Code D&D Manger Device Info Appliances' Info Mobile Phone 4 22:

41 4 32 VisualCode ID UDN 0x a <UDN>uuid: a </UDN> 4 1: VisualCode ID UDN UI DataFlowController UPnP A 3 B UI UI getname() UI displaydragsearchmenucontent() displaydragsearch- MenuConfig() displaydragselectmenucontent() displaydragselectmenuconfig display- DropSearchMenuContent() displaydropsearchmenuconfig() executeassociation() UI # i f n d e f SERVICEAPP H # d e f i n e SERVICEAPP H 4.1: ServiceAppUI.h # i n c l u d e <c o e c n t r l. h> # i n c l u d e A p p l i a n c e I n f o. h / / CLASS DECLARATION c l a s s C A p p l i a n c e I n f o ;

42 4 33 / * * * ServiceAppUI * * / c l a s s MServiceAppUI { p u b l i c : / / v i r t u a l TDesC& getname ( ) { } ; v i r t u a l void setname ( ) { } ; / / v i r t u a l TDesC& getdragfunctionname ( ) { } ; v i r t u a l TDesC& getdropfunctionname ( ) { } ; / / v i r t u a l TBool d i s p l a y D r a g S e a r c h M e n u C o n t e n t ( ) { } ; v i r t u a l TBool displaydragsearchmenuconfig ( ) { } ; / / v i r t u a l TBool d i s p l a y D r a g S e l e c t M e n u C o n t e n t ( ) { } ; v i r t u a l TBool d i s p l a y D r a g S e l e c t M e n u C o n f i g ( ) { } ; / / v i r t u a l TBool d isplaydropsearchmenucontent ( MServiceAppUI * dragserviceappui, TDes& AssocName ) { } ; v i r t u a l TBool displaydropsearchmenuconfig ( MServiceAppUI * dragserviceappui, TDes& AssocName ) { } ; / /

43 4 34 v i r t u a l TBool e x e c u t e A s s o c i a t i o n ( C A p p l i a n c e I n f o * d r a g D e v i c e I n f o, C A p p l i a n c e I n f o * d r o p D e v i c e I n f o, TDesC& AssocName ) { } ; p r i v a t e : } ; # e n d i f / * SERVICEAPP H * /

44 Nokia N Sony Type X 4 5 PC

第5章ユーザスタディ 36 図 5 2: 実験環境その 2 図 5 1: 実験環境その 1 5.

ドロップを元にして家電を操作できることを述べた後に実験を開始するコンテンツ連携ハードディスクレコーダーからテレビ番組を選択しディスプレイで再生するコンフィグレーション連携正常な時計のデータを送信し

45 第5章ユーザスタディ 36 図 5 2: 実験環境その 2 図 5 1: 実験環境その実験手順まず最初に被験者に実験の狙いを説明し実験内で想定する以下の 2 種類のシナリオについて解説するその後被験者にシステムに関する簡単な説明を行う本システムがドラッグドロップを元にして家電を操作できることを述べた後に実験を開始するコンテンツ連携ハードディスクレコーダーからテレビ番組を選択しディスプレイで再生するコンフィグレーション連携正常な時計のデータを送信し時刻がずれる時計を定期的に同期させる 5.4 評価方法被験者へのインタビュー項目は本実験で行ってもらう作業を日常どれくらい行っているかという質問と実験の感想に関する質問の二種類である日常の作業の頻度について PC 上などで普段ドラッグドロップを行うかいつどのような時に行うか右ドラッグドロップは行うか日常生活の中で家電連携を行うことはあるか

46 5 37 GUI PC Alt /6 2/6 1/6 1/6 5 1: PC

47 /6 2/6 2/6 5 2: GUI GUI UI 2 GUI

48 VisualCodes CyberLink 90 ID RFID IC 2

49 40 6 ID RFID UPnP GUI

50 41 Mr. Fahim Kawsar Mr. Vili Lehdonvirta Ambient Intelligence

51 42 [1] R.T. Azuma, et al. A survey of augmented reality. Presence: Teleoperators and Virtual Environments( ), Vol. 6, No. 4, pp , [2] IT. e-japan [3] Y. Iwasaki, N. Kawaguchi, and Y. Inagaki. Touch-and-connect: a connection request framework for ad-hoc networks and the pervasive computing environment. Pervasive Computing and Communications, 2003.(PerCom 2003). Proceedings of the First IEEE International Conference on, pp , [4] N. Kohtake, J. Rekimoto, and Y. Anzai. InfoStick: An Interaction Device for Inter- Appliance Computing. Handheld and Ubiquitous Computing: First International Symposium, HUC 99, Karlsruhe, Germany, September 1999: Proceedings, [5] J. Rekimoto. Pick-and-drop: a direct manipulation technique for multiple computer environments. Proceedings of the 10th annual ACM symposium on User interface software and technology, pp , [6] J. Rekimoto, B. Ullmer, and H. Oba. DataTiles: a modular platform for mixed physical and graphical interactions. Proceedings of the SIGCHI conference on Human factors in computing systems, pp , [7] M. Rohs and B. Gfeller. Using camera-equipped mobile phones for interacting with real-world objects. Advances in Pervasive Computing, pp , [8] B. Shneiderman. 1.1 Direct manipulation: astep beyond programming languages. Sparks of Innovation in Human-Computer Interaction, [9] B. Ullmer, H. Ishii, and D. Glas. mediablocks: physical containers, transports, and controls for online media. Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pp , [10] Life. elife. Technical report,, [11].. Technical report,,

52 43 [12],. transtick:. The 11th Workshop on Interactive Systems and Software, [13].. Technical Report 1,, [14],.,, 2004.

2) 2. DLNA DLNA (Version 1.5) 2 (DMC1) (SSDP) (DMS1, DMS2) (DMR1, DMR2, DMR3) (UDP) DMC1 3 DMS2 DMC1 DMS1 (HTTP) DMS1 DMR2 (RTP) DMR2 3. DLNA 4 DMC1 D

2) 2. DLNA DLNA (Version 1.5) 2 (DMC1) (SSDP) (DMS1, DMS2) (DMR1, DMR2, DMR3) (UDP) DMC1 3 DMS2 DMC1 DMS1 (HTTP) DMS1 DMR2 (RTP) DMR2 3. DLNA 4 DMC1 D 1 1 2 2 DVD LED A Networked Home Appliance Control Method Using Augmented Reality Kenya Sato, 1 Akira Sakamoto, 1 Shinya Mihara 2 and Hideki Shimada 2 Many kinds of networked home appliances connected