From Snow White and the Magic Mirror To Scientist and Multimedia Retrieval
What was happening there? Mr. Scientist, one night, was traveling to Fairy-tale land
O Lady Queen, Snow-White is the fairest of them all Cool! How the mirror can do that? Mirror, mirror, on the wall, Who is the fairest of them all?
Searching Process Searching Sending question with invocation 呪文 Asking question Selecting answer... Sending result Showing result
Snow-White's stepmother: User who creates the question or the query 白雪姫の継母 質問 クエリ をつくる Mirror: Interpreter who translates muggles language to wizard language, and vice versa 魔法の鏡 魔法を使えない人間の言葉を魔法使いの言葉に訳したり その逆をする Wizard: Seeker who uses his magic search engine to find the right person in the fairy-tale land. The answer is called retrieval 魔法使い 魔法のサーチエンジンを使っておとぎの国の住人の中から クエリにあった人を探す これを検索 retrieval と呼ぶ
Mr. Scientist started to think about it. He wants to turn the magic mirror into his computer and turn the wizard into search engine
Remember Computer only understands 0 and 1 All data must be transformed to 'digital signal' デジタル信号
Translate from picture to digital image
Translate from sound to digital audio signal
すべて解決したように思うけど It seems that everything is solved, but...
Challenges The question of Snow-White's stepmother is who is the fairest of them all?. It means that, the magic mirror must look for The most beautiful girl, and The most benevolent girl 親切な
Challenges コンピュータはどうやって 美しさ を評価するのか How to evaluate a beauty? for example, in fairy-tale land, a beautiful girl must reach a standard Hair is as black as ebony color 色 Body is as slender as willow shape 形 Skin is as soft as silk texture 質感 Singing voice is as nightingale s signal 音 Since these information belong to a girl, it is 内部特徴 called internal 内部情報 information or internal features
Challenges How to evaluate a benevolence? for example, in fairy-tale land, a benevolent girl must reach a standard Be loved by most of inhabitants in fairy-tale land 慈愛心から Doing a lot of things out of charity Since these information does not belong to a girl, it must be collected from inhabitant who 外部特徴 know the girl, it is called external information. The features could be letters, audio/video records, books, etc.
Some technical terms Color, texture, shape, and signal are called low-level features that are extracted directly from objects and the computer very easily understands. 色 質感 形 音は低レベル特徴量と呼ばれる これは 画像 映像中の モノ から直接取り出すことができて コンピュータでも理解しやすい
Some technical terms Hair, body, skin, and eyes are called highlevel features. In order to let the computer understand these features, Mr. Scientist must prepare a lecture by which the search engine could recognize them. 色 質感 形 音は低レベル特徴量と呼ばれる これは 画像 映像中の モノ から直接取り出すことができて コンピュータでも理解しやすい That lecture is called bridge the semantic gap between low-level and high-level features 教えてあげる ことを低レベル特徴量から高レベル特徴量への セマンティックギャップの橋渡しという
Some technical terms After the computer translate query of users to features, the search engine will compare these features to features in its database to choose the best match one. コンピュータ 魔法の鏡 が質問を特徴量に変換したら サーチエンジン 魔法使い がデータベース内の特徴量と比べて一番似ているものを選ぶ The tool to calculate/compare the similarity between two sets of features is called Distance measure. 特徴量同士がどれだけ似ているかを比べるには 距離尺度を使う
Remind Remind... おさらい One object usually has internal features and external information Multimedia includes images, audio, video, and text Query is what users ask computer to search Retrieval is what search engine found
Remind Remind... Computer does not understand human being language. Computer easily recognizes lowlevel features but hardly understand highlevel features The semantic gap between low-level features and high-level features is large, and it is very difficult to bridge this gap
Remind Remind... Search engine only understands and works with features Distance measure is necessary to comparing two sets of features Browsing is the process of showing results returning by search engine under human languages 閲覧は人に理解しやすい形で検索の結果を見せること Indexing is the process of re-ordering data in database in order to look for faster 索引付けは見つけやすくなるようにデータを並び替えること
Query by keyword Wrong result
Annotation Manual Time consuming FALSE Cinderalla Mermaid OK Automatic Fast but very difficult
Automatic annotation
Query by concepts
スケッチ
Query by color It looks good... But sometime the results are not good at all
キーワード
History of Search Engine's development
Conclusions Beginner Intermediate Expert