MediaTeam Oulu

Background and Mission

MediaTeam Oulu (MediaTeam), founded in 1997, is a research group of about 50 people at the Information Processing Laboratory in the Department of Electrical and Information Engineering at the University of Oulu. MediaTeam is a multidisciplinary research group, having researchers with backgrounds in electrical and information engineering, computer science, mathematics, linguistics and architecture.

MediaTeam’s mission is to carry out leading-edge long-term research, which produces new scientific knowledge and novel technological solutions for mobile, distributed multimedia and communications.

MediaTeam conducts research on the features, use, and applications of multimedia and digital media types (image, sound, video, text) in information and communication systems. MediaTeam’s research combines the different areas of information and telecommunications technology, with a special focus on mobility and wireless features as well as future generations of communication technology. MediaTeam’s main fields of interest are distributed computing, image and video processing, information hiding, language and audio technology, mobile services and packet networks.

Scientific Progress

The three-year Duchess – Wireless Media Telephony Services in the Office Environment project, funded by the National Technology Agency and industry concluded in March 2002. The project focused on the research and development of new methods utilising media telephony and their application particularly to a modern office environment. The primary goal was to implement a multimedia telephone architecture according to the latest proposals in packet-based multimedia communication standards and the signalling standards H.323 and SIP. The central results include Beethoven1 and Beethoven2 Internet telephony architectures, a media telephone system developed with the architecture, and new information hiding algorithms. A media telephone system operating in an internal office network was implemented with the help of the Beethoven architectures, offering services such as real-time multimedia conference and remote control of applications. For information hiding, the project produced scientifically valuable and comparable methods and results for both images and audio. The algorithm developed for audio is applicable for both watermarking (copyright protection) and secret messaging. An information hiding service, in which the algorithm is applied to secret messaging during a multimedia conference, was implemented.

The ongoing Red Skins – Mobile Multimedia & IP Telephony Fabric project, funded by the National Technology Agency and industry, focuses on mobility management, Internet-telephony security and software techniques enabling the development of distributed interactive communications services. Research topics include development of Mobile IPv6 handoff-methods, error-resiliency of IPT-applications and distributed computing, especially in lightweight computing environments. The main results achieved so far include a new hybrid model for mobility management in IPv6 networks, a method for improving distribution performance in slow wireless environments and a test environment for empirical evaluation of fourth generation applications and protocols. Also, research on IPT application and SIP protocol implementation security and resilience against buffer-overflow attacks has been conducted. A Jini-based service platform has been developed which utilizes integrated context-information module and the delivery of multimedia services to PDA devices. A context information module has been developed to ease context-aware application development by moving a part of the context information logic to the server module. The Red Skins project will continue to May 2003.

The three-year-long CAPNET – Context-Aware Pervasive Networking research program, funded by the National Technology Agency and industry, started in January 2002. It is one of the four national spearhead initiatives funded by the NETS – Networks of the Future technology program. The goal of the CAPNET program is to create a foundation for new information and communications technologies and for business in the field. The focus is on context-aware mobile technologies for ubiquitous computing. These are technologies, which allow communication anytime and anywhere, with any kind of terminal device, automatically taking into account the characteristics of the network and the terminal. CAPNET consists of four major segments, which are 1) diversity management, 2) adaptive software structures, 3) adaptive content techniques and 4) adaptive user interface techniques. In CAPNET research problems and technology requirements are derived from actual usage needs and scenarios arising from real human needs. Application scenarios have been generated to form a foundation for architecture and technology development work. The CAPNET reference architecture providing the basis for implementing context-aware and pervasive systems and their components has been specified. The basic entities of a CAPNET system are engines, which each specialize in producing a certain type of facility: service discovery, user interface, context recognition, media processing, connectivity management, component management, database access, etc. The CAPNET program will continue to the end of year 2004.

Technologies developed in the CAPNET program are used to create new interactive and smart mobile services which adapt to suit each situation according to the user profile and resources offered by the usage environment.

The Stardust – Mobile Programmable Network Techniques and Mobile Economic Modeling project, funded by the National Technology Agency and industry, started in January 2002. The project is conducted jointly by MediaTeam and the Economics unit at the Faculty of Economics and Industrial Management. The purpose of the project is to research the mechanisms of electrical mobile commerce and the economic models related to protection of digital contents as parts of future network and application services, as well as new service concepts of mobile telecommunications. These include superdistribution, wireless ATM’s and payment mechanisms, along with light downloadable applications as integral elements in creating new mobile services. The focus of the research is on new service mechanisms using complex revenue logic networks and on the modelling of network transactions, particularly in view of economic operating models and technologies supporting digital rights management.

The Stego – Hiding secret information in audio and video recordings started in March 2002. The project, funded by the National Technology Agency and industry, focuses on developing methods of digital watermarking and the use of cryptography in digital rights management (DRM). DRM is used to protect digital content from illegal copying and distribution. The encrypted content can be safely transmitted to customers, but it does not protect the content after it is decrypted by the user. Watermarking can be used to complement the cryptography by protecting the content after decrypting it. During the first year a number of digital watermarking algorithms for images and music were developed. The algorithms concentrate on two areas: robust watermarking and steganography. The former is used to embed a watermark that cannot be removed by a malicious user and the latter is used to hide as much data in the content as possible. A software platform for testing DRM tools in different kinds of scenarios was also created.

Watermarking can be used to hide information in cover media. The middle image can be hidden in the first one, and as a result we get the last image. Later the hidden information can be extracted.

The mGain – Mobile Entertainment Industry and Culture project, funded by the EU under the Information Society Technologies (IST) Program, started in September 2002. The project focuses on various aspects of mobile entertainment, paying attention to technological, commercial, cultural, legal, and social features. The research aims at creating analyses and surveys of the mobile entertainment industry and culture for use by EU authorities, the mobile entertainment industry and the scientific community. Particularly the needs and expectations of mobile entertainment users are considered in the research, emphasising service usability and pricing along with ethical and socio-economic issues. The main topics of research are to benchmark the European situation with North America and Asia Pacific, to draw up guidelines for the industry and political decision-makers, and to prepare mobile entertainment services and technologies within the 6th Framework Programme for research in the EU. The results will benefit not only developers of new entertainment services but also various authorities, whose tasks include issues related to consumer protection and content regulation. MediaTeam is responsible for mapping and analysing the technological area of mobile entertainment for the project. The analysis includes searching for the technological possibilities and, on the other hand, also the shortcomings that affect the development of the mobile entertainment market in Europe.

MediaTeam’s research program on content-based multimedia retrieval is realized in two ongoing research projects: Vikings – Content-based Video Retrieval System for Distributed Environments, funded by the National Technology Agency and industry, and Semantic Gap - Filling of the Semantic Gap in Information Retrieval, funded by the Academy of Finland. There are two new research projects funded by the Academy of Finland, which started in January 2002: CBIR - Content-based information retrieval and Prosody of Emotions - Multiparametric prosodic analysis of phonetic and phonological correlates of emotions. The Content-based Mobile Multimedia Retrieval project funded by the Academy of Finland was completed in the end of the year 2002.

This program investigates multimedia content analysis and its application in content-based video and audio retrieval systems. The main objective is to develop methods for narrowing down the "semantic gap" between the concept-based and content-based approaches to database indexing, and the utilization of these methods in practical retrieval applications. Filling the gap is important in order to enable the design of databases and search engines where it is easier to map user-specified search criteria to off-line/on-line computed index terms and the meta-data of the database. The problem is especially unresolved as regards forthcoming media types like digital speech, music, image, and image sequence, where the search criteria often include semantic concepts. The problem is tackled with a cross-disciplinary approach involving information engineering, linguistics, natural sciences, and information studies. The main results achieved on audio/video retrieval systems include:

1) VIRE video retrieval system. It is based on cluster-temporal browsing, a novel approach for interactive browsing and retrieval of large video databases. The approach combines the traditional time-line presentation of a video shot with that obtained by unsupervised multi-modal content-based clustering with self-organizing maps into a single representation of the video database. The resulting 2-D view into the video database allows efficient interactive navigation in two semantic spaces, temporal adjacency and content similarity, simultaneously. The view can be refined with optional semantic filtering, which corresponds to detecting the presence of any designated semantic concept(s) in the video data represented to the user. The experimental results obtained at the international TREC-2002 Video Track benchmark showed that VIRE provides an efficient approach for realizing interactive video retrieval. Additionally, VIRE supports manual query-based retrieval of video shots.

User interface for computer-aided content-based video browsing in the VIRE (Video browsing and Retrieval) system.

2) New speech signal processing algorithms were developed for searching speech corpuses. The algorithms compute in excess of 40 prosodic features of speech. Three data sets were collected including speech samples of several persons speaking in 5–7 different emotional states. Data clustering was done by discriminant analysis and multi-dimensional scaling functions. Results show that a good separation of emotional states of human speakers can be achieved with statistical pattern recognition methods relying on multi-dimensional feature spaces. Results are comparable to human performance achieved with listening tests of the same material.

An example of statistical emotions discrimination: the groups consisting of points describing different voice samples which represent emotions, such as hate, joy, sorrow and neutral emotion.

3) A new sound-profiling tool was developed that segments sound signal to speech, music and mixed parts and displays results graphically. Statistical classification of sound can be performed with a very degree of accuracy. A simple browser was implemented which enables browsing an audio database and showing sound profiles of recordings.

Exploitation of Results

The results have been disseminated to the academic community in the form of about 40 scientific publications. MediaTeam also places lots of emphasis on popularisation of science, which has resulted in dozens of articles in major Finnish newspapers, some international newspapers, professional magazines and periodicals, teletext of the Finnish Broadcasting Company and on-line news archives.

A concrete example of exploitation of results has been the MFO Showroom located at the Science Centre Tietomaa, which was closed in December 2002. It was an exhibition presenting the research, products and services of mobile information and communication technology with prototypes and models: the different generations of devices for wireless and mobile communications from heavy ARP phones to GPRS technology were well represented. One of the missions of the MFO Showroom was to present to the general public the research projects and parties involved in the Mobile Forum program, creating visions, prototypes, and implementation models for the mobile communication technologies and services of the future.

Future city life might look like this: a tourist navigates to the services in the Oulu town centre with the help of a wireless terminal. Although this currently requires a PDA device, in a few years’ time a mobile phone may suffice. In the centre of Oulu, a PDA device is now also a map and more. Test services can be found and used with a small terminal like this one.

International and domestic collaboration

MediaTeam values close collaboration with its partners as an essential factor in conducting leading edge research and in transferring research results into practical solutions. The most important partners are the following funding bodies, companies, and research organisations:

The National Technology Agency, the Academy of Finland, CCC, Hantro, IBM, Jutel, Nokia, OPOY/Finnet Group, Sonera, Yomi Fusion, VTT Electronics, Mobile Forum, University of Maryland (USA), Linköping University (Sweden), and various research groups and laboratories at the University of Oulu.

In December 2002 MediaTeam organized in cooperation with the ACM SIGGRAPH, MUM 2002 – The 1st International Conference on Mobile and Ubiquitous Multimedia, in Oulu, Finland. Infotech Oulu was the main sponsor of the conference. The conference attracted over seventy participants from around the world. Additionally, MediaTeam hosted several international experts giving a talk in the Infotech Oulu Lecture Series: Prof. Henning Schulzrinne (Columbia University, USA), Dr. Oskar Juhlin (Interactive Institute, Sweden), Prof. Steven Feiner (Columbia University, USA) and Peter Marx (Binary Protocol, USA).

Future Goals

MediaTeam will continue to strive towards becoming one of the leading authorities in multimedia research. MediaTeam will pursue its goals of providing practical, easily adaptable solutions to the multimedia application industry and of performing eminent research in its fields of expertise. MediaTeam will remain in active contact with industry as a whole, as well as with its existing partners, and it will strengthen and develop its international network. MediaTeam will further improve its innovation process to ensure an even more efficient manner of transforming research results into the practical applications.

The above goals are to be realized in ongoing research projects and forthcoming projects currently under preparation. We have participated in the build-up of several international research projects in the scope of EU’s 6th Framework Programme. Similarly, we have been preparing several large national applied research projects, which will seek funding from the National Technology Agency. As in the past, long-term basic academic research projects will be funded by the Academy of Finland, one of our more significant sources of funding.

One specific and important goal in the scope of multimedia analysis is successful participation in the TRECVID 2003 competition. It is the premier international benchmark in multimedia retrieval, bringing together the world’s leading research groups in the field. The purpose of the competition is to promote progress in content-based multimedia retrieval via open and well-defined evaluation.

Personnel

External Funding

Selected Publications

Howie D (2002) Consequences of using MIPv6 to achieve mobile ubiquitous multimedia. Proc. 1st International Conference on Mobile and Ubiquitous Multimedia, Oulu, Finland, 34-39.

Keskinarkaus A, Cvejic N, Niskanen A, Seppänen T & Sauvola J (2002) Improvements on watermarking images with m-sequences. Proc. International Workshop on Digital Watermarking, Seoul, Korea, 333-344.

Mäntyjärvi J & Seppänen T (2002) Adapting applications in mobile terminals using fuzzy context information.
Proc. Fourth International Symposium on Human Computer Interaction with Mobile Devices, Pisa, Italy, 95-107.

Ojala T & Ollila M (editors) (2002) Proceedings of the 1st International Conference on Mobile and Ubiquitous Multimedia, Oulu, Finland, ISBN 951-42-6909-8, 112 p.

Ojala T, Pietikäinen M & Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with Local Binary Patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7): 971-987.

Rautiainen M & Doermann D (2002) Temporal color correlograms for video retrieval. Proc. 16th International Conference on Pattern Recognition, Quebec, Canada, 1: 267-270.

Rautiainen M, Penttilä J, Vorobiev D, Noponen K, Väyrynen P, Hosio M, Matinmikko E, Mäkelä SM, Peltola J, Ojala T & Seppänen T (2002) TREC 2002 Video Track experiments at MediaTeam Oulu and VTT. Proc. Text Retrieval Conference TREC 2002 Video Track, Gaithersburg, MD.

Seydou F & Seppänen T (2002) A simplified Newton method in scattering theory: The TE case. Proc. IEEE AP-S International Symposium and USNC/URSI National Radio Science Meeting, San Antonio, TX, 2: 102-105.

Sun J & Sauvola J (2002) On fundamental concept of mobility for mobile communications. Proc. 13th IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, Lisbon, Portugal, 2: 799-803.

Toivanen J & Seppänen T (2002) Prosody-based search features in information retrieval. Proc. Fonetik 2002, Stockholm, Sweden, 105-108.

Väyrynen P, Seppänen T, Noponen K & Juuso I (2002) On the usefulness of linguistic knowledge in different areas of application in language technology. Informaatiotutkimus 3/2002: 59-66 (in Finnish).