University of Oulu
INFOTECH OULU

Intelligent Systems Group (ISG)

Professor Juha Röning, Professor Jukka Riekki and Professor Tapio Seppänen,
Computer Engineering Laboratory, Department of Electrical and Information Engineering, University of Oulu

juha.roning(at)ee.oulu.fi, jukka.riekki(at)ee.oulu.fi, tapio.seppanen(at)ee.oulu.fi

http://www.ee.oulu.fi/research/isg


Background and Mission

The Intelligent Systems Group's (ISG) mission is to carry out long-term research on novel technologies and applications of intelligent systems. The main objective is to develop enhanced adaptivity and context-awareness for smart environments. The research specifically focuses on the creation of dynamic models that enable monitoring, diagnostics, prediction and control of target systems (living and artificial) or operating environments. It is our aim to make the environment adapt to the users, instead of making the users adapt to an inflexible environment. We believe that by creating these novel components for smart environments, important enabling functionality will emerge that will multiply the versatility and applicability of such living environments.

We see behaviour modelling as a major challenge in developing truly intelligent and proactive environments. Human users of smart environments often behave in such a complex manner that it is hard to predefine and pre-program all of their behavioural patterns in the software. Models of user behaviour are required that are able to grasp the user's context at any moment, and to enable adaptation of the functionality of the intelligent environment to the situation at hand. Further, it is essential to model the behaviour of the devices controlled by the intelligent environment, as this enables adaptation to environmental changes without reprogramming. Systems should eventually learn and adapt automatically through these models to perform their duties effectively.

Our research group combines a variety of key skills and technologies to aim at this goal. We have experience in the following key technologies: system architectures and implementation of context-aware systems; modelling and recognition of contexts from sensor signals; data mining algorithms; learning nomadic robots; embedded systems technologies; software security; and smart environment implementations. The key application areas are: smart living environments in homes and institutes; industrial automation; mobile robots; context-aware mobile devices; and wellness and medical applications. Each of these domains possesses special characteristics, but, from the point of view of developing algorithms for an intelligent system, they also possess remarkable similarities. They all produce a multitude of signals that represent the status of the system. The target system behaviour should be modelled and recognized from the signals. The application service should then act accordingly. The availability of several application domains yields many advantages: a solution to a special problem in one domain may offer added-value functionality in some other domain; our solutions are deployed by many of our client industries; solutions to a wide range of real-world problems define a credible and versatile tool-box that has a major impact on our development-oriented sub-contractual projects.

The group co-operates with many international and domestic partners. In applied research, the group is active in European projects. The group hosted three large European project meetings (ROBOSWARM, XPRESS, eConfidential) in 2008. In addition, several joint projects are funded by the Finnish Funding Agency for Technology and Innovation (Tekes) and industry.

The group and its members are active in the scientific community. For example, Prof. Juha Röning co-chaired numerous international workshops in the software security area: The 9th Winter School of the European Intensive Programme on Information Security Management and Technology (IPICS) was arranged in Rovaniemi, and the 4th Crisis Management Workshop (CRM 2008) in Oulu, Finland. In information security, the group acted as a member of the SAFECode International Board of Advisors, 2008. The group also coorganized a Vulnerability Prevention and Software Security seminar with a keynote speech by Mr. Howard Schmidt in April, 2008.

Prof. Tapio Seppänen is a leading figure in both the WellTech Oulu Institute, and the Oulu School of Biomedical Engineering, an umbrella organization of wellness and medical technology education at the University of Oulu and the Oulu University of Applied Sciences.

Several members of the group were also on the committees of international conferences. The group's expertise is recognized, testified by the many invited talks and lectures that have been given. The Intelligent Systems Group has communicated its research to the public and its research areas have attracted interest in the media.


Scientific Progress

In 2008, the research at ISG concentrated on prototyping smart environments, mobile and context-aware systems, data mining methods, signal analysis and secure programming. These were applied in context-aware mobile systems, intelligent service robots, quality control of steel plant and spot welding processes, and analysis of biomedical ECG and EEG signals.

Research on Prototyping: from a Smart Environment towards Remote Distributed Intelligence

Verification of the developed methods and models in prototypes will be an important part of the research. To support this activity, we will develop software and hardware architectures for smart environments. In addition to verification, prototypes speed up the commercialization of the research results. In prototyping, we have set and tackled the following objectives:

Developing hardware architecture for a smart environment. The aim is to develop a basis for devices operating in a smart environment. Modular technologies have been the subject of our research for some years now. Modularity has been achieved using a modular electronic concept, the Embedded Object Concept (EOC) Atomi and a modular software architecture - Property Service -, which provides an interface for these resources, and an opportunity to control different robots and other devices of a smart environment.

EOC is a concept that utilizes common object-oriented methods used in software by applying them to combined Lego-like software-hardware entities. These modular entities represent objects in object-oriented design methods, and they function as the building blocks of embedded systems.

This concept enables one to build new embedded systems from electronic Lego-like building blocks. The goal of the EOC is to make designing of embedded systems faster and easier while preserving the commercial applicability of the resulting device. The EOC enables people without comprehensive knowledge in electronics design to create new embedded systems. For experts, it shortens the design time of new embedded systems. Implementing the conceptual idea of embedded objects has been successfully implemented with the Atomi II framework.

The EOC research has proceeded by developing new objects. The focus of the research has been on improving their robustness, and testing the concept in practice by supporting other research projects. The research will continue towards user friendly development tools in order to further improve the concept.

Research on Mobile Robotics

Localization is one of the fundamental problems in mobile robotics, as in many applications a robot needs to know its location in order to perform its tasks. A global self-localization technique was proposed that utilizes observations of the ambient magnetic field. This study was inspired by evidence that animals use the magnetic field of the Earth for true navigation. The experiments reported in this article suggest that (especially) modern buildings with reinforced concrete and steel structures have unique spatially varying ambient magnetic fields that can be used for navigation, in very much the same way as the Earth's magnetic field, but on a smaller scale. In principle, a non-uniform ambient magnetic field produces different observations, depending on the path taken through it. The approach provides a promising and simple technique for solving the global indoor self-localization problem.

The current technique is applicable for a one-dimensional localization problem, i.e., for localizing a robot or a person within corridors. However, the proposed approach could be extended to two- or three-dimensional localization problems, assuming that maps can be provided. In some applications, the proposed approach may provide an alternative to machine vision based approaches, especially when only one-dimensional localization is needed, or when the illumination of the environment changes. On the other hand, the proposed technique may also be used in parallel with machine vision and range finder based approaches in order to overcome possible sensor aliasing problems.

The experiments suggest that the ambient magnetic field may remain sufficiently stable for longer periods of time. In the conducted experiments, the magnetic field remained nearly unchanged, although some variations were observed. These variations did not, however, have a great impact on the localization performance. On the other hand, the magnetic field is not sensitive to many environmental changes that may affect other localization techniques, such as vision or range finder based techniques which rely on visual or geometrical features. For example, the magnetic field is not sensitive to non-magnetic dynamic or static objects, nor to changes in illumination.

 

Two-dimensional magnetic map in a corridor of the Computer Engineering Laboratory, and the measurement system used in global self-localization experiments.

 

Multi-robot exploration methods can be utilized to explore unknown environments using a team of robots. In ISG, a novel multi-robot exploration algorithm has been studied. The exploration algorithm chooses frontier cells for each individual robot so that the overall exploration time is minimized. The frontier cell is a cell between a known and an unknown area. Whenever a robot arrives at the frontier cell, new information is received around the cell. Each cell of the occupancy grid map contains a probability that the cell is occupied. The cell is occupied if the corresponding area in the environment is covered by an obstacle. The only requirement is that the map must allow the distinction between known and unknown areas and it must compute travel costs for the individual robots. The algorithm simultaneously takes into account the cost of reaching a frontier cell and its utility. The cost of the cell depends on the occupancy probabilities of the cells along the path of the robot, and the distance between the robot and the cell. Whenever a frontier cell is assigned to a robot, the utilities of the nearby cells are reduced. The utility of the frontier cell also depends on the number of robots moving near to the cell. Our results suggest that this coordinated multi-robot exploration technique is more effective than uncoordinated techniques, as is shown below.

Paths of the robots in a coordinated case; the occupancy map produced by a team of coordinated robots; paths of the robots in an uncoordinated case; and the corresponding occupancy map.

 

Path optimization is a very important aspect of multi-robot systems in which the available resources, such as battery life, can be limited. Many real-world multi-robot path optimization problems can be restated as an instance of the Multiple Travelling Salesman Problem (MTSP). A novel ant colony based algorithm called TACO was proposed for solving the MTSP with a min-max objective. Its competitiveness was shown by comparison to neural network based approaches, and its feasibility in multi-robot systems was demonstrated in a simulation environment.

 

TACO generated paths for two, three, and four robots in a hospital environment (top), and for three robots in the Computer Engineering Laboratory (bottom).

 

In autonomous environment modelling research we sense the environment using mobile robots. This enables selection of optimal sampling locations in order to provide a model with maximum accuracy. In geostatistics studies, optimal spatial sampling has traditionally focused on the selection of sampling locations in advance. However, with mobile sensors, we are able to select the locations based on the current model which increases the accuracy of the model and decreases the measuring time.

On the top: an experimental environment including 1) a machine vision localization system, 2) measurement area, 3) battery charger and 4) central computing unit. At the middle are integrated mean squared errors for stationary and non-stationary spatial models as a function of samples for ambient light and magnetic flux density, respectively. At the bottom are regression results after 50 samples for these quantities.

 

We have developed mobile sensing systems for various environments, including indoor, outdoor, and aquatic environments. In indoor environments, the most important measured quantities are temperature, humidity, and gas concentrations. In aquatic environments, measurements on dissolved oxygen, pH levels, and temperature provide important information for modelling these environments. Our research integrates the latest results from geostatistics, and multi-robot systems to enable implementation of fully autonomous, and adaptive sensing systems.

A prototype of a robot boat for sensing aquatic environments: 1) an embedded computer with a WLAN communication interface, 2) sensors, and 3) GPS positioning system.

 

Research on Context-Aware Services

Context-aware services adapt to the user's situation. We have studied context recognition in several different application areas, including human physiology, wellness, urban computing, smart office spaces and body sensor networks. The aim is to find and develop methods for recognizing the user's context from sensor data using signal processing, pattern recognition and machine learning methods. Context can be used to identify the services that are relevant in the situation at hand - and to adapt these services.

During the year 2008, the research on context recognition has continued with gesture recognition. We have developed novel solutions and applied Hidden Markov models to enable gestural controlling with an ordinary mobile phone. The gestures are recognized from 3D acceleration measurements that the N95 or Nokia Sport 5500 can produce, but also other 3D acceleration sensor devices are applicable. With the gestures, different applications can be controlled, including the inner functionality of the mobile phone. The gesture controller is configured with (gesture, command, application) triplets. When the controller recognizes a gesture, it sends the corresponding command to the specified application. The system architecture is presented in the following figure. The user can teach the gestures and associate each gesture to a controlling command. For example, a user can select a clockwise circle drawn in the air to correspond to the command for opening the calendar application on an N95. The second figure shows a user controlling with gestures an application on a wall display. The software was published as open source on November 2008 and it can be downloaded from the group's web site.

The gesture controller.

 

A user controlling an application on a wall display with gestures.

 

During the last year the group also studied recognition of context from ambient audio. Here, audio from users' everyday environments was recorded using a mobile device, and through signal processing and classification, it was used as a cue of the users' current context. Part of the work was done in collaboration with Nokia Research Center and MIT, and funded by the Fulbright organization.

The group is strongly moving on to studying signal processing in networked sensing systems. The group members are participating the UbiCity project, where a tool on gathering, storing and processing sensor measurement from a heterogeneous sensor network is being built. The group has participated in organizing a data mining workshop on sensorwebs, databases and mining in networked sensing systems (2nd SWDMNSS 2008) together with Japanese and German researchers. This workshop and several two-way research visits with Tokyo Denki University's Ubiquitous Networking Laboratory have created international collaboration.

In addition to recognizing a user's context, sensor data can be used to build a physical user interface in which a user uses a mobile device as a physical object rather as a traditional I/O device. The device is equipped with sensors and the actions of the user are recognized from the sensor data and interpreted as commands. Gesture controlling is an example of a physical user interface; the user commands the system by waving the terminal on the air. In addition to gestures, the group is actively developing touch-based physical user interfaces. These user interfaces are based on RFID technology. An RFID tag storing service parameters is placed behind an icon advertising a service. When a user touches the icon with a mobile device that is equipped with an RFID reader, the service parameters are read from the tag and delivered to the system. Touch-based user interfaces are an effective way of controlling a system, as they produce rich contextual information: who requests a service, which service, when, and where. With such interfaces, the user remains in control.

During 2008, the group developed several innovative touch-based user interfaces in co-operation with the University of Lapland. The group also developed further the REACHeS platform that can be used to link physical user interfaces to Internet services. The figure below presents a touch-based user interface for controlling videos on a wall display. At the bottom left we see an icon that advertises a service in the environment, next to a wall display. When a user touches this icon with a mobile device, a video is shown on the wall display (top of the figure) and a remote control UI is created on the mobile device's display (bottom right). The users can now control the video on the wall display using their mobile device Two alternative physical user interfaces for the same service have been developed: Control Panel and Control Cube.

 

A physical user interface for controlling videos on a wall display.

 

In addition, we created an application for sharing information at our university's Zoological Museum. In this application, icons are placed next to exhibition items, the stuffed animals. Touching an icon with a mobile device brings a list of item-related documents to the device's display. The user can download the selected files to a device and for example, listen the sound of an animal, or study a photograph. Over 300 students from local primary and secondary schools tested the application in spring 2008.

Research on Data Mining

Biosignal Processing

Cardiovascular signal processing. Sudden cardiac arrest is the most common cause of death in western countries. It accounts for approximately 50% of cardiovascular deaths, and apparently has a highly variable pathophysiological etiology. The risk of sudden cardiac arrest is high in certain subgroups of patients with a history of myocardial infarction and depressed left ventricular function. The key question in research is why do some subjects develop ventricular fibrillation during acute coronary occlusion, while others survive this episode without fatal arrhythmia. The challenge for research is to develop approaches or techniques that will allow the screening of the specific risk for fatal ventricular arrhythmias as a predictor of the first event in patient populations that have a low cumulative risk, but generate a large number of victims. In addition, the predictive value of many known risk factors of sudden cardiac arrest among patients with known heart disease has not been definitively established.

Invariant trajectory classification of dynamical systems with a case study on ECG. An invariant pattern recognition framework for classification of phase space trajectories of nonlinear dynamical systems was developed. Using statistical shape theory, known external influences can be discriminated from true changes of the system. The external effects are modelled as a transformation group acting on the phase space, and variation of the trajectories not explained by the transformations is accounted for using principal component analysis. The approach suggested is highly adaptable to a wide range of situations and individual differences. The methodology presented is applied to detect abnormalities in electrocardiograms. Results based on measured data indicate that the model developed is resistant to the effects of respiration and body position changes, which are abundant in ambulatory conditions and cause significant morphological artifacts in the signal. The results also show that the detection of an artificially induced acute myocardial infarction is achieved with high performance. Due to its low computational complexity, the method developed can be implemented in real-time. This method also adapts to morphological changes caused by various heart conditions.

A method for estimating the severity of myocardial infarction from multi-channel ECG. The measurement of infarct severity is an important element in the overall care of patients with STEMI (myocardial infarction with ST segment elevation) patients. The severity of the MI (myocardial infarction) is commonly estimated by cardiac biomarker methods and echocardiography. We developed a method for estimating the severity of the MI by combining an action potential based computer model and 12 lead ECG patient data. The estimators of the severity of the anterior and inferior MI were developed using an EDL (equivalent double layer) model. The best combinations of the single parameter based estimators were found by using multiple linear regression analysis. The correlations between the final estimators and two clinical estimators of the severity of the MI were calculated. The severity estimators correlated to the maximum troponin value with r value 0.615 and to the ejection fraction with r value 0.428. On the grounds of the results, it is possible to calculate a coarse estimate for the maximum troponin value, and therefore the severity of the MI, from the standard 12 lead ECG by using the simulation model based method. The estimated troponin value can be used to yield a fast assessment of tissue damage of an ambulatory cardiac patient that is suspected to have an infarction.

Improving Reliability of `Total-Cosine-R-to T' (TCRT) in Patients with Acute Myocardial Infarction. The parameter TCRT (Total Cosine R-to-T) calculated from ECG recording has been shown to have a remarkable prognostic value as a predictor of the outcomes of coronary artery disease and acute myocardial infarction (AMI) patients. The TCRT is conventionally calculated using an algorithm produced by Acar et al. (1999). In this study, the reliability of the TCRT algorithm was tested with the ECG data of a healthy group (n = 25) and the AMI group (n = 45). Typical problems occurred in the detection of the maximum of the T vector (9% of patients), the bounding of the R wave (18%), a comprehensive segmentation (11%), and a decreased congruence between TCRT and the spatial QRS-T angle (33%). The results show that small improvements to the basic algorithm can decrease the number of failures by up to 82% in AMI data. It is concluded that segmentation properties should be improved in the basic TCRT algorithm in order to maintain the diagnostic value of TCRT in different patient data.

EEG signal processing. The aim of this task is to develop new methods and algorithms for estimating the depth of anaesthesia from a multi-channel EEG. A new concept of relative induction time was recently developed by our group, which enables a significant improvement in the depth estimation from EEG. The new method produces a time-continuous value for the depth estimate and accurately predicts the instant of loss of consciousness. The method was extended so that it can model the depth of anaesthesia to such a depth that burst-suppression of EEG starts to evolve. The team has so far focused on describing the spectral characteristics of EEG related to the induction of anaesthesia with propofol. The time-frequency characteristics of EEG that are consistent between individuals during induction of anaesthesia were determined. The relationship between the EEG frequency progression pattern and the clinical end-points, such as loss of obeying verbal command, was shown. A mathematical method was developed for describing the consistent EEG frequency progression pattern during anaesthesia. The method has provided us the possibility to study in detail the effects of analgesic drug (remifentanil) on the EEG-based depth of anaesthesia estimation. The results indicate that remifentanil modifies the relationship of EEG spectral changes and clinical endpoints in propofol anaesthesia. The effects of remifentanil on the onset of burst suppression pattern were also presented.

EEG activity in different frequencies during induction of propofol anaesthesia.

 

Text production speedup technologies. The feasibility of word prediction was studied in a highly inflected language, Finnish, where words are used in many case forms, a topic seldom addressed in the context of word prediction. Since about one third of words will appear in uninflected forms in Finnish, simple prediction methods, e.g. word completion, typically employed in uninflected languages such as English can be used as such in the prediction of uninflected words. The preliminary results obtained show that about 45% of characters can roughly be saved in Finnish word prediction in general for uninflected words. Secondly, the utility of predicting entire phrases instead of single words, (as usually done in word prediction), consisting of two or more words was investigated in English word prediction. The results obtained show that about 70% of characters of the phrases included in a lexicon of some 7,000 phrases of legal English could be saved in theory when using the best search key for their prediction.

Data Mining Systems

The focus areas of data mining research were re-organised during the year 2008, and the research is now carried out with a quite unique approach. The research challenges are divided into three mutually supportive categories; the research of algorithms producing knowledge, software running the algorithms and knowledge bases storing the acquired knowledge. When put together, these three categories form a strong combination which can be applied to virtually any phenomena where data can be processed into knowledge.

The research on algorithms was focused on advancing methods for time series analysis, variance modelling and novelty detection. Special interest was directed to establishing data driven methods for these areas, where the exact shape and nature of the observed data can be used to characterize the phenomena under study. In software research, the implementation of the first version of new software architecture for running the algorithms came to successful conclusion. The architecture presents the algorithms as information generating logical devices to which the information measuring physical devices connect. A suitable combination of logical and physical devices can thereafter be used to form a data mining software application. In knowledge base research, the general ideas behind the role of knowledge bases were established, and the implementation of the first version of a concrete knowledge base was carried to a successful end.

Three focus areas of the data mining research; algorithms generating knowledge, software running the algorithms and knowledge bases storing the acquired knowledge.

 

The research has focused on three projects, XPRESS (2007 - 2010), SAMURAI (2006 - 2008) and MIDAS (2008 - 2010). XPRESS is an integrated (IP) EU-project, where new approaches for managing and optimizing the operation of entire production facilities are being developed. In 2008, data mining algorithms were created, for example, for recognizing the tools the associates in the production lines are using at a particular time. The analysis is based on accelerometer data, and it can help, for example, in finding more ergonomic ways of working. The software architecture and knowledge bases were applied to storing the generated knowledge in this application and also a wider range of applications in factories.

The SAMURAI-project was completed during 2008, and the results were evaluated to be so good that they gave rise to the starting and granting of new funding for a project, MIDAS, which is based on the results. In both projects, methods for utilizing data originating from a steel factory and human being were developed. The algorithms applied to steel production helped in reducing the variability of the quality of steel plates. Also, methods for pointing out the right time to perform a production model parameter update were reported. In the analysis of the human data, methods for estimating the energy expenditure of various sports exercises based on accelerometer data were reported for the first time. Also, research for recognizing sport activities from accelerometer time series was continued successfully.

The group had collaboration in the focus area of data mining both internationally and nationally. The researchers gave presentations of their results in 10 conference presentations around Europe, the USA and Asia. Co-operation with European researchers and industry was intensive also thanks to the XPRESS project in which 17 different European organizations are participating. The data mining group also belongs to the Centre of Advanced Steel Research, which is founded to gather together from both at home and abroad steel research expertise at the University of Oulu.

At the end 2008, the future for data mining research seems bright, and major advancements in all of the three focus areas (algorithms, software and knowledge bases) can be expected in 2009. International collaboration can be expected to deepen even more during the next year thanks to forthcoming long term research visits established in 2008.

In 2008, data mining methods were applied for example to monitoring tool usage of associates in production lines. The method could be used to improve worker ergonomics.

 

Research on Software Security

Within the Intelligent Systems Group, the Oulu University Secure Programming Group (OUSPG) has continued research in the field of implementation level security issues and software security testing. Software implementation may introduce potential for unanticipated and undesired program behaviour, e.g. an intruder can exploit the vulnerability to compromise the computer system.

In 2008, the research focus at OUSPG remained on black-box methodology for improving software security. OUSPG approaches the problem from three different directions, namely, network traffic data-mining and visualization, protocol genes and protocol dependence.

Causal relationships. The aim is to develop methods for inferring causal relationships in complex systems. The research applied data mining to network traffic to find data relevant to the system being analysed. Applications of the method include getting an overall view of the communication patterns of complex systems, diagnostics and security risk assessment.

Information network environments are more complex than ever before, and the complexity will increase in the future. One important factor affecting the increase in the complexity is the convergence of information networks. One seemingly simple event may generate a number of small actions in different parts of the network. Additionally, different components may use varying protocols. Thus the secure development, deployment and management of complex networks is laborious and requires in-depth understanding of different fields.

Network traffic data mining research applies black-box methodology in understanding the behaviour of different components in computer networks. Examples of systems where this work has been applied are operator WLANs, firewalls and malware. Operator WLANs are a good example of complexity due to the large amount of infrastructure involved (e.g. integration with traditional phone network authentication infrastructure). Firewalls may have unintentional information leaks due to misconfiguration or implementation problems. Malware is software that is purposely obfuscated to make reverse- engineering and removal difficult. By monitoring the external behaviour of malware, it is possible to understand how it functions.

Identification of protocol genes. This research, PROTOS-GENOME, approaches the problem of complexity from the other direction, by developing tools and techniques for reverse-engineering and identification of protocols based on using protocol genes - the basic building blocks of protocols. The approach is to use techniques developed for bioinformatics and artificial intelligence. Samples of protocols and file formats are used to infer structure from the data. This structural information can then be used to effectively create large numbers of test cases for this protocol.

The PROTOS-GENOME project produced test suites for several archive file formats that are typically implemented, for example, in anti-virus products. The test suites have been distributed to vendors in cooperation with FICORA and NISCC. A number of vulnerabilities was found in widely used antivirus software products. The method has been also applied to find plagiarism in text.

Structural inference methods used in plagiarism detection.

 

As a continuation of OUSPG's practical security research, work was conducted on the security of RFID systems. This work is on-going in 2009.

Vulnerability management of the information infrastructure contributes to protocol dependence. Another angle on battling complexity is through technology dependencies. This activity studies the impact factor of different technologies on CNI, and develops a visual model for understanding dependencies related to protocols. This was accomplished by extending traditional Wikis, which are effective mass collaborative authoring services with graphing extensions. Graphingwiki enables deepened analysis of the Wiki data by augmenting it with semantic data in a simple, practical and easy-to-use manner. Visualisation tools are used to clarify the resulting body of knowledge so that only the data essential for a usage scenario is displayed. Logic inference rules can be applied to the data to perform automated reasoning based on the data. Perceiving dependencies among network protocols presents an example use case of the framework.

Practical RFID security research in progress.

 

A newly established area of research in OUSPG is computer-assisted teaching. This work originated in 2005 with RAIPPA, software for automated software for programming laboratory pre-exercises. Work on RAIPPA continued in 2008 with funding from Campus Futurus, an organization within the university that promotes the use of ICT in teaching. The system has improved learning results significantly, and it is currently being piloted with courses from throughout the university.


Exploitation of Results

The results of our research were applied to real-world problems in many projects, often in collaboration with industrial and other partners. Some examples of exploitation are described below. Especially during the reported year, outdoor robotics was a new area for exploitation of our research results.

The Intelligent Systems Group utilizes a robotics laboratory and pressure-sensitive floor (EMFi material) installed in our laboratory as part of a smart living room. Other equipment includes a home theatre, two degree-of-freedom active cameras, four mobile robots and one manipulator, a WLAN network, and various mobile devices (PDAs, a tablet PC, Symbian mobile phones). WLAN positioning covers a large part of the campus (including the laboratory), and a home automation network is being installed. Our aim is to gradually build a versatile infrastructure that offers various generic services for pervasive applications. Naturally, this kind of environment enables realistic experiments that lead to a better understanding of such applications.

The embedded objects implemented according to the Atomi II Framework specification are called Atomi objects. Several different Atomi objects have been created for real life tests. The Atomi objects have been used in several projects, and they have proved to be very usable.

The Atomi objects are being used both in pure research projects and in projects that aim ultimately at commercial products. The Atomi objects have been applied to telepresence robots, nanoscale manipulation and measurement technology, a hand held medical device, and several robot applications. The most recent applications include, for example, a micromanipulation platform and a bio film measurement device.

The micromanipulation platform (shown below) is a device platform that can be used for different applications that require actuation and sensing in nanometer resolution. Presently, nanoactuation devices on the market are very expensive, and often limited in applications. This platform is mostly built with off-the-shelf components and Atomi objects, and thus brings about a reasonable cost for the instrument.

Micromanipulation platform.

 

The device is based on a generalized modular architecture which covers both device hardware and the control software in a PC. The modular architecture enables a swift changing of the actuators, sensors and tools with minimal effort and reusable source code, thus being an ideal frame for various applications.

The platform consists of a haptic 3D controller, a piezoelectric actuator device, a probe and a PC. The haptic 3D controller enables manual control over the robot for the user. The controller has a pen-like handle which can be moved with six dimensions of freedom. The pen has two buttons which can be assigned to different functions. The controller also provides a haptic feedback option that can be used to let the user feel what the robot feels.

The piezoelectric actuator device consists of three linear piezo stages with position encoders. This actuator moves the measurement. The device can be easily changed. We have used a SmarAct piezo actuator, which can move in a resolution of 5 nm in 3 cm range.

The probe (measurement head) is connected to the actuator. It can be customized to every device separately. In our test case, the probe is a silicon strain-gauge force sensor AE-801, which is attached to mechanically custom designed arm. The probe is connected to a voltage amplifier, and further to AD-conversion on 24-bit precision AD converter Atomi with USB and Power Atomi objects. A specially designed needle is attached to the force sensor, to feel the surroundings. In some applications the probe can include an additional actuator. This actuator can be, for example, a micro gripper, which also can be controlled via Atomi objects.

A bio film measurement device measures the resistance of a thin soft coating i.e. the bio film. This device is used in measuring the properties of different type of coatings for certain purposes. The thickness of the coating varies between 1 µm and 2 mm. During the measurement, the coating is on a level surface in a bowl filled with water. The resistance is measured between a probe needle and the surface that the coating is located on. The surface is a plane made of material that conducts electricity. The needle is 10 µm in diameter.

The measurement is not simple. In order not to damage the bio film, the measurement current must be very low and its polarity must constantly change. Otherwise the sensitive surface may burn or otherwise get damaged. Furthermore, the measurement must be made with several points on the bio film in a reasonable amount of time. These requirements demand an intelligent measuring method and thus the programmable Atomi objects are very well suited for the job.

The next figure shows the setup of the device. It consists of several Atomi objects, the piezo actuators, the measurement probe and a PC for the high level control and a UI for the system. The Atomi objects that are in this device include two PiezoLegs Atomi objects, a USB Atomi, a power Atomi and an AD-DA converter measurement Atomi.

The results of the multi-robot and distributed sensing research will be utilized in a real world application scenario as a part of the ROBOSWARM project, which is an EU project that was launched in November 2006. Together with eight other participants, the University of Oulu has an important role in this project, which aims to develop an open knowledge environment for self-configurable, low-cost and robust robot swarms usable in everyday applications. Advances in the state-of-the art of networked robotics are proposed through the introduction of a local and global knowledge base for ad hoc communication within a low-cost swarm of autonomous robots operating in the surrounding smart IT infrastructure. For the ROBOSWARM project ISG has developed a custom embedded control system for the swarm of robots. The Sensor and Connectivity Board (SCB) was designed with the aim of provide a modular robot platform for swarm robotics research and application development, where simplicity and modularity are key-factors. The SCB is used to integrate all sensor modalities and the motor control into one logical device, which can be accessed through a USB port from any computer having USB host capability. This solution providing easy access to all sensors, and flexibility for selecting an embedded host computer to perform the high-level data processing, and motion control computing. The sensor set currently supported by the SCB includes an RFID reader, sonars, a laser range finder, 3-axis accelerometer, gyroscope, photo diode, thermometer, humidity and atmospheric pressure sensors, 3-axis magnetometer, and a real-time clock. SCB also features a built-in interface to iRobot's Create mobile robot, but it can easily be connected to any robot platform with the UART connection. In addition, SCB provides I2C, RS232, and SPI interfaces for adding new external devices to expand the functionalities of the SCB even further.

An instrumented Robotic Team Member (RTM) built for the ROBOSWARM project.

 

During the reporting year, the group continued utilizing outdoor robotic systems. Development and utilization of Mörri, a multipurpose, high performance robot platform continued, and major components for software architecture were implemented. The software architecture further developed the earlier work of Property Service Architecture. The main focus of development has been on multi-purpose control architecture, that can be used to integrate various algorithms, methods and sensors to one real-time system. Development of the robot included platform mechanic design, high power brushless motor control electronics and software, sensor integration, robot world modelling and route reasoning user interfaces. The user interface is designed for field use, including wearable control devices, and use of Google Earth software for visualizing and controlling the robot's global path.

The Group participated in the Military European Land Robot Trial 2008, held in Hammelburg Germany in July. M-Elrob is the biggest outdoor robot event in Europe, and participants are research facilities and companies that represent the state-of-the-art in Europe in this research area. From four scenarios of competition, Mörri succeeded in two, by winning the Camp Security scenario, and getting fourth place in the Mule transportation scenario. In October, the same Mörri platform was used in the European Space Agency's Lunar Robot Challenge (Esa-LRC) held in Teide, Tenerife. A new application module was implemented, including a sampling device for soil samples. The task in competition was to get samples from a 15 m deep crater with up to a 40 degrees slope. Conditions in the challenge were temperature -2 degrees Celsius, hard wind and total darkness (without ambient light). Our robot was third in the competition and won world-wide publicity in newspapers, and on radio and TV.

Mörri platform performing in European Space Agency's Lunar Robot Challenge in Teide, Tenerife.

 


Future Goals

We will continue to strengthen our long term research and researcher training. We will also continuously seek opportunities for the exploitation of our research results by collaborating with partners from industry and other research institutions in national and international research programs and projects. The group is a founding member of the European Robotic Network of Excellence (EURON). The group is a contract member of EURON II which was approved for the EU's FP6 as a Network of Excellence.

We will strengthen our international research co-operation. Within the last years, the group has created collaboration projects with Japanese researchers. Several researchers have visited Waseda University's Distributed Computing Laboratory (DCL) to study ubiquitous computing paradigms. The collaboration has given rise to results on sentient (recognizing the user of an artefact), as well as activity recognition and the corresponding architectural and data collection issues. With the University of Tianjin, China we have a joint project in which methods and a system will be developed for vision-based navigation of Autonomous Ground Vehicles, which utilize an omni-directional camera system as the vision sensor. The aim is to provide a robust platform that can be utilized in both indoor and outdoor AGV (Autonomous Ground Vehicles) applications. This co-operation will continue.

In the USA, we will co-operate with the Human-Computer Interaction Institute in Carnegie Mellon University. A doctoral student from ISG will make a one year research visit to the institute and co-operate with assistant professor Anind D. Key. The research will be on human modelling in the area of human machine interaction.

A new co-operation agreement has been sealed during year 2008 with IDIAP research in Switzerland. A post doctoral researcher from ISG will make a one year research visit starting March 2009. Shorter research visits to European partners of EC funded projects are also planned.


Personnel

professors & doctors

8

graduate students

27

others

42

total

77

person years

55

External Funding

Source

EUR

Ministry of Education

282 000

Tekes

225 000

other domestic public

42 000

domestic private

154 000

international

532 000

total

1 235 000


Selected Publications

Askola K, Puuperä R, Pietikäinen P, Eronen J, Laakso M, Halunen K & Röning J (2008) Vulnerability dependencies in antivirus software. Second International Conference on Emerging Security Information, Systems and Technologies (SECURWARE '08), Cap Esterel, France, 273-278.

Fujinami K & Riekki J (2008) A case study on an ambient display as a persuasive medium for exercise awareness. Lecture Notes in Computer Science: Persuasive Technology, Springer, 5033: 266-269.

Halunen K, Rikula P & Röning J (2008) On the security of VSH in password schemes. Third International Conference on Availability, Reliability and Security (ARES 2008), Barcelona, Spain, 828-833.

Juutilainen I & Röning J (2008) Modelling conditional variance function in industrial data: A case study. Statistical Methodology, 5(6): 564-575.

Kemppainen A, Mäkelä T, Haverinen J & Röning J (2008) An experimental environment for optimal Spatial Sampling in a Multi-Robot System. Intelligent Autonomous Systems 10 (IAS-10), Baden Baden, Germany, 54-63.

Koivikko M, Perkiömäki J, Karsikas M, Salmela P, Tapanainen J, Ruokonen A, Seppänen T & Huikuri H (2008) Effects of controlled hypoglycemia on cardiac repolarization in type I diabetes. Diabetologia, 51: 426-35.

Kortelainen J, Koskinen M, Mustola S & Seppänen T (2008) Remifentanil modifies the relation of electroencephalographic spectral changes and clinical end points in propofol anesthesia. Anesthesiology, 109: 198-205.

Kortelainen J, Koskinen M, Mustola S & Seppänen T (2008) Time-frequency properties of electroencephalogram during induction of anesthesia. Neuroscience Letters, 446: 70-4.

Koskimäki H, Juutilainen I, Laurinen P & Röning J (2008) Detection of Correct Moment to Model Update. Lecture Notes in Electrical Engineering: Informatics in Control, Automation and Robotics, Springer Berlin Heidelberg, 24: 87-94.

Peltola M, Tulppo M, Kiviniemi A, Hautala A, Seppänen T, Barthel P, Bauer A, Schmidt G, Huikuri H & Mäkikallio T (2008) Respiratory sinus arrhythmia as a predictor of sudden cardiac death after myocardial infarction. Annals of Medicine, 40: 376-382.

Pietikäinen P, Viide J & Röning J (2008) Exploiting Causality and Communication Patterns in Network Data Analysis. 16th IEEE Workshop on Local and Metropolitan Area Networks (LANMAN 2008), Cluj-Napoca, Transylvania, Romania, 114-119.

Riekki J, Sánchez I & Pyykkönen M (2008) Universal Remote Control for the Smart World. Lecture Notes in Computer Science: Ubiquitous Intelligence and Computing, Springer, 5061: 563-577.

Röning J, Haverinen J, Kemppainen A, Mörsäri H & Vallivaara I (2008) Smart Systems for Distributed Sensing. Proceedings of 11th Biennial Baltic Electronics Conference (BEC2008), Tallinn, Estonia, 21-30.

Schaberreiter T, Wieser C, Sánchez I, Riekki J & Röning J (2008) An Enumeration of Rfid Related Threats. Proc. The Second International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies (UBICOMM’08), Valencia, Spain, 381-389.

Seydou F, Duraiswami R & Seppänen T (2008) Numerical solution of electromagnetic scattering by multiple cylinders. ACES Journal, 23.

Siirtola P, Laurinen P & Röning J (2008) A Weighted Distance Measure for Calculating the Similarity of Sparsely Distributed Trajectories. Seventh International Conference on Machine Learning and Applications, San Diego, USA, 802-807.

Suutala J & Röning J (2008) Methods for Person Identification on a Pressure-sensitive Floor: Experiments with Multiple Classifiers and Reject Option. Information Fusion Journal, Special Issue on Applications of Ensemble Methods 9(1): 21-40.

Tamminen S, Juutilainen I & Röning J (2008) Product Design Model for Impact Toughness Estimation in Steel Plate Manufacturing. The International Joint Conference on Neural Networks (IJCNN 2008), Hong Kong, 990-993.

Tiinanen S, Tulppo M & Seppänen T (2008) Reducing the effect of respiration in baroreflex sensitivity estimation with adaptive filtering. IEEE Transactions on Biomedical Engineering 55: 51-9.

Vallius T & Röning J (2008) Low Cost Arbitration Method for Arbitrarily Scalable Multiprocessor Systems. 4th IEEE International Symposium on Electronic Design, Test and Applications (delta 2008), 119-124.

Noponen K, Kortelainen K & Seppänen T (2008) Invariant trajectory classification of dynamical systems with a case study on ECG pattern recognition. Pattern Recognition. In print.

Pirttikangas S, Fujinami K & Hosio S (2008) Experiences on data collection tools for wearable and ubiquitous computing. Int. Symposium on Applications and Internet (SAINT2008), Workshop on SensorWebs, Databases and Mining in Networked Sensing Systems (SWDMNSS 2008), IEEE, July 28, Turku, Finland, 149-152.