Bayesian Machine Learning Algorithms for Discovering Relations with Latent Variable Models

Tapani Raiko's Academy of Finland postdoctoral researcher's project 2010-2012

Adaptive Informatics Research Centre, Department of Information and Computer Science, Aalto University School of Science and Technology

The project is about developing graphical models that can find new types of latent sources from data. These sources describe relationships between observations instead of describing the observations directly, as in traditional methods. Graphical models are a class of statistical machine learning methods that combine graphs as a model structure with probability theory for inference on the variables at the nodes of the graph. Latent variable models such as independent component analysis, are graphical models that explain the patterns in observed data by latent sources. In the proposed methodology, the activity of a source can indicate for instance a correlation between some of the observations.

Applications for the proposed methods are done in close collaboration with domain experts. The collaboration includes robotics with the ZenRobotics company, climate modelling using the NCEP/NCAR Reanalysis data with Alexander Ilin, texture analysis in content based image retrieval, speech recognition in noisy environments, analysis of complex time series in collaboration with Xerox Research Centre Europe, and collaborative filtering in social networks such as Last.fm.

During the project, the research focus has drifted slightly towards deep learning.

Figure on top right: The model structure for drifting linear dynamics. The continuous valued hidden state s(t) affects the linear dynamics of the observation sequence x(t).