Standards Standards

Multimedia Metadata Standards

Metadata is the value-added information which documents the administrative, descriptive, preservation, technical and usage history characteristics associated with resources. It provides the underlying foundation upon which digital resources management systems are based to provide fast, precise access to relevant resources across networks and between organizations.
Multimedia content analysis refers to understand semantic meanings of MM documents by means of metadata extracted using common techniques of image and signal processing and image analysis and understanding.
Metadata is an important aspect of the creation and management of digital images and other MM files. The information contained in metadata standards can regards the following aspects:

the technical format of the image file
the process by which the image was created
the content of the image

Following these standards helps organizations to consistently record information about their MM documents in a way that facilitates retrieval and sharing in a networked environment.
Metadata for MM documents can be classify according to the following three types:

Descriptive or Content metadata: is information about the object captured in the document (the object's name, title, materials, dates, physical description, etc..). Content metadata is very important, as it is the main way by which people can search and retrieve the MM documents from a database. There are standards available to assist in determining what information should be recorded about the object, and how to record it.
Technical metadata: is also essential to properly manage digital images. Technical metadata is data about the MM document itself (not about an object in the document). For example, for a digital image, it can include information about: the technical processes used in image capture or manipulation or colour or file formats, and some of the technical information that is recorded about the image, such as the image file type, must be machine-readable (following specific technical formats) so that, a computer system will be able to properly display the image.
Administrative metadata includes information related to the management of MM documents (such as rights management).

MM Metadata can be also classified according to other criteria considering the level of data description, the producibility and the domain dependence [4].
They can be classified by:

Level: we can distinguish between a technical level in which lower level aspects of the MM content is described and a semantic level in which aspects of higher level of abstraction on the MM content are taken into account.
Producibility: the production of metadata can either be automatic, which is a very desirable property from the economic point of view, and regards more frequently the low level technical metadata; for semantic metadata describing the information covered by MM content it is typically required human knowledge. So in these cases the metadata production is manually performed.
Dependencies: metadata can be domain-dependent, for instance the position of a tumour can be interesting for medical applications, while the colour distribution of an image can be useful for many application domains. Metadata can also be media type-dependent considering for instance the colour distribution as applicable only to visual media while the creation date applicable to any media.

Metadata represents surely a gain in terms of benefits produced for MM data descriptions but most of all for MM applications (content analysis).
There are also disadvantages related to metadata. Some of them are its cost, its unreliability, its subjectivity, its lack of authentication and its lack of interoperability with respect to syntax, semantics, vocabularies and languages. However, there are many researchers currently investigating strategies to overcome different aspects of these limitations in an effort to provide more efficient means of organizing content on the Internet.
The main reference topics related to techniques and projects developed for MM content analysis can be briefly summarised as follows [3]:

Automatic document Indexing/Classification: there is a variety of techniques to classify documents in subject categories. These techniques include: Bayesian analysis of the patterns of words in the document, clustering of sets of documents according to similarity measures, neural networks, sophisticated linguistic inferences, the use of pre-existing sets of categories and seeding categories with keywords. The most common methods used by auto-categorization software are based on scanning every word in a document and analysing the frequencies of patterns of words and, based on a comparison with an existing taxonomy, assigning the document to a particular category in the taxonomy. Other approaches use “clustering” and “taxonomy building” techniques searching through all combinations of words to find cluster of documents that appear to be together. Some systems are capable of automatically generating a summary of a document by scanning through the document and finding important sentences using rules like: the first sentence of the first paragraph is often important.
New researches are focusing on Semantics-Sensitive Matching [1] and Automatic Linguistic Indexing in which the system is capable of recognizing real-world objects or concepts [2]. Image retrieval research has moved on from the IBM QBIC (query by image content) system (QBIC, 2001) which uses colours, textures, and shapes to search for images [7]. In particular the IBM CUEVideo project [50]
Speech recognition is increasingly being applied to the indexing and retrieval of digitised speech archives. Speech recognition systems can generate searchable text that is indexed to time code on the recorded media, so users can both call up text and jump right to the audio clip containing the keyword. Normally, running a speech recogniser on audio recordings does not produce a highly accurate transcript because speech-recognition systems have difficulty if they haven't been trained for a particular speaker or if the speech is continuous. However the latest speech recognition systems will work even in noisy environments, are speaker-independent, work on continuous speech and are able to separate two speakers talking at once.
Video Indexing and retrieval: the latest video indexing systems combine a number of indexing methods, embedded textual data, scene change detection, visual clues and continuous-speech recognition to convert spoken words into text. Some systems [51] can automatically analyse videos and extract named entities from transcripts which can be used to produce time and location metadata. This metadata can then be used to explore archives dynamically using temporal and spatial graphical user interfaces.
The Annotation systems also represent an useful tool for metadata extraction. The motivation behind annotation systems is related to the problem of metadata trust and authentication. Users can attach their own metadata, opinions, comments, ratings and recommendations to particular resources or documents on the Web, which can be read and shared with others. The basic philosophy is that we give more probably value and trust to the opinions of people we respect than metadata of unknown origin. The W3C's Annotea system [54] and DARPA's Web Annotation Service [49] are two web-based annotation systems which have been developed. Other annotation tools for film/video and MM content (IBM VideoAnnEx, 2001) [52], (Ricoh MovieTool, 2002) [53], (DSTC's FilmEd, 2003) [56] and tools to enable the attachment of spoken annotations to digital resources (PAXit, 2003) [55] such as images or photographs have been developed.
Metadata for Preservation : a lot of initiatives are focusing on metadata pursuing the goal of the MM resource preservation. Such initiatives include: Reference Model for an Open Archival Information System (OAIS, 2002) [57], the CURL Exemplars in Digital Archives project (CEDARS, 2002) [58], the National Library of Australia (NLA) PANDORA project (PANDORA, 2002) [59]. These initiatives rely on the preservation of both digital objects and associated metadata for an easy interpretation in the future. The preservation metadata provides sufficient technical information about the resources and can facilitate the long-term access of the digital resources by providing a complete description of the technical environment needed to view the work, the applications and version numbers needed, decompression schemes as well as any other files that need to be linked to it.

A large number of metadata standardisation initiatives has been developed in recent years, in order to describe MM contents in so many different domains and to grant sharing, exchanging and interoperability across wide range networks.
We can distinguish between two different standard typology, according to what each of them represents in terms of its functionalities.
The first typology is directly related to the representation of MM content for a specific domain and each of these standards can be referred as a standardised description scheme, while the second one considers the possibility of integrating more metadata standards mapped on different application domains, providing rich metadata models for media descriptions together with languages allowing one to define other description schemes for arbitrary domains.
The table below represents a selection of several metadata standards description schemes, which can be considered the most frequently cited and representative for a quite wide range of different application domains. So, in the following table is illustrated a list of descriptive characteristics of each reference standard taken in account. In particular, information about standardisation bodies, last version dates, described MM data types, application domains, description semantic levels and the way by which metadata has been produced (manually or automatically) have been considered.

	MARC	Dublin Core	CDWA	VRA Core	CSDGM	Z39.87	LOM	DIG35	METS	JPX	SMPTE Metadata Dictionary
Standardization Body	Library of Congress	Dublin Core Metadata Initiative (DCMI)	Art Information Task Force (AITF)	Visual Resource Association	Federal Geographi Data Committee (FDDC)	National Information Standard Organization (NISO)	IEE (LTSC)	Digital Imaging Group (DIG of I3A)	Digital Library Federation (DLF)	Joint Photographic Experts Group (JPEG)	Society of Motion Picture and Television Engineers (SMPTE)
Year	Current Version MARC 21 since 1999	Current Version 1.1 since 1999	Current Version 2.0 since 2000	Current Version 3.0 since 2002	1998	2002	2002	Current Version 1.1 April 2001	Last Review 2001	2000	Last Review 2004
MM Type	Any	Any	Any	Images	Any	Images	Any	Images	Any	Images	Any
Domain	Bibliographic media description	Bibliographic media description	Description of Art works	Description of images of Art works	Description of Geographic media	Description of still images	Description of educational media	Description of digital images	Description of digital objects	Description of digital images	Description of audio/video objects
Level	Largely semantic	Largely semantic	Largely semantic	Largely semantic	Semantic and technical	Technical	Largely semantic	Semantic and technical	Semantic and technical	Semantic and technical	Semantic and technical
Producibility	Mainly manual	Mainly manual	Mainly manual	Mainly manual	Manual and Authomatic	Mainly automatic	Mainly manual	Mainly manual	Mainly manual	Mainly manual	Manual and Authomatic

Selection of several standardised description schemes