Chương 1: Các khái niệm cơ bản MIRS Issues
Nguyễn Thị Oanh Bộ môn HTTT – Viện CNTT & TT oanhnt@soict.hut.edu.vn
1
Architecture
Main operations – Insert new item – Retrieval
2
Main operations
3
Data Model
Data model determines :
Requirements: DM should be
– How information is organized and stored – What types of queries are supported
relationships
– extensible, new data type can be added – able to represent basic media type + temporal, spatial
at different levels of abstraction
– flexible so that items can be specified, queried, searched
4
– allow efficient storage and search
A General Multimedia Data Model
OO-based Multiple layers: – Object layer
Spatial relationship: window size + position for each item Temporal specification: timeline-based: start time + duration
Common media type Features or attributes for each media type are specified ex.: image: size, color histogram, main objects contained
– Media type layer:
Specifies the media formats for proper encoding, analysis & presentation
5
– Media format layer
A General Multimedia Data Model
6
Data Model: Remaining issues
Each layer:
Most MIRS: application-specific
– Not completely designed – No common standard
– Limited number of features – Limited number of data type
- VIMSYS: image + video - a general video model - virage image schema structure
7
Special data model for each application:
Data Model: Example
VIMSYS (Visual Information Management System)
Define events that can be queried
User-defined entity:
1 concept (sunset ) or a physical entity (heart)
Segmentation layer (temporal, spatial info, ..) + Feature layer: histogram, texture
Data + transformation (compression, color space conversion, image enhancement, …
8
User interface
Requirements:
allow user to :
– Insert database items easily – effectively and effeciently enter queries – Present query results to the user effectively and efficiently – Be user-friendly
9
– specify various types of input – compose multimedia objects – Specify attribute types to be extracted and indexed
User interface – Query support
Multimedia query:
– Diverse – Fuzzy ==> tools:
By keywords, parameters mapping problem: « red car » By example need input tools: microphone, camera, …
– Searching:
A very large query Based on the DB organization Item randomly chosen
– Browsing: start browsing with
10
– Query refinement: Feedback
User interface – Result presentation
Many design issues:
+ QoS
– Present all media types + temporal, spatial relationships
long audio segment,
information to large
long video,
browse for: image…
– How to extract and present essential
11
– Reponse time should be short (communication subsystem time + DB search time) – Felicitate relevance feedback and query refinement
Feature extraction
Determine the retrieval effectiveness Requirements:
information items
– complete as possible to represent the contents of the
12
– represented and stored compactly – The computation of distance between features should be efficient, otherwise the system response time would be too long
Feature extraction
Levels of feature
Example
Handling Techniques
Meta data
DBMS
Author name, date, title, …
Information Retrieval
Text annotation (captures abstract concepts)
Content description, keywords: happy, sad, …
Content-based retrieval
Audio: frequency distribution, … Image: color distributions, texture,shapes, …
Low-level (data patterns and statistics of a multimedia object, and possibly spatial and temporal relations between parts of the object)
recognize and interpret humans
Content-based retrieval
High-level (attempts to recognize and understand objects)
13
Indexing
1 Object ~ many features 1 feature ~ many parameters
Indexing in MIRSs should
Application classification Different levels of features spatial and temporal relationships between objects
14
– be hierarchical and – take place at multiple levels.
Similarity measurement
Similarity: computed on extracted features retrieval Relevance of
results:
judged by human
(subjective and context dependent)
? Computed similarity values should be conform to
human judgement – Features used ? – Similarity measure used ?
15
QoS (Quality-of-Service)
MMData requires: – High bandwidth – large storage space and high transfer rate – delay and jitter bound – and temporal and spatial synchronization
key components:
the operating system – the storage manager – the transport or communications system
16
– hosts (including clients and servers) under the control of
Tổng kết
Data model User interface: query support + presentation Feature extraction Indexing Similarity Measurement Storage
17
18
