Courses in previous years: [ 2000 | 2001]

Tik-122.101 Special Course in Information Technology

Graphical Models

Graphical Models - course description

Graphical models are probabilistic models, which have origins in many different research communities, such as artificial intelligence, statistics and neural networks. The general framework of graphical models provides a mathematical formalism, that helps in understanding similarities and differences between various learning architectures and algorithms. Typically, algorithms in a particular network architecture can be derived from the inference and learning machinery of general graphical models. On such example is the junction tree algorithm, which can be used to derive inference rules for the Gaussian mixture models and hidden Markov models, for instance. The course will cover topics around Bayesian networks, Boltzmann machines, mean-field and variational approximations to learning, latent variable models and propagation of probabilities in networks with loops. The material is based on a recent textbook (see above), which is a collection of original contributions of leading researchers in the field.

The course is suitable for advanced graduate students and students with a specific research focus on graphical models. The course has great thematic overlap with the course Inference and Learning in Bayesian Networks organized last year (in Finnish), so think carefully before enrolling. Attendance will be limited to 15-20 students.

To pass the course, students are expected to give an oral presentation on one of the given topics (chapters) and hand out a summary to others. In addition to active seminar participation, there will be exercises and a programming exercise. Exercises must be returned by January, 31st, 2003.

The course information is presently missing from Topi. You can enroll by by sending e-mail to the course assistant and by attending the first seminar on September 18th.

Timetable

Date	Subject	Speaker	Material
18.9.	Introduction	Jaakko Hollmén
25.9.	1 Probabilistic Independece Networks for Hidden Markov Probability Models	Salla Ruosaari
2.10.	NO SEMINAR
9.10.	2 Learning and Relearning in Bolzmann Machines (+ 3 Learning in Boltzmann Trees)	Jussi Pakkanen
	4 Deterministic Boltzmann Learning Performs Steepest Descent in Weight-Space	Jussi Ahola
16.10.	8 Variational Learning in Nonlinear Gaussian Belief Networks	Merja Oja
23.10.	11Hierarchical Mixtures of Experts and the EM Algorithm	Jaakko Hollmén
	5 Attractor Dynamics in Feedforward Neural Networks (+ 6 Efficient Learning in Boltzmann Machines Using Linear Response Theory	Eerika Savia
30.10.	12 Hidden Neural Networks	Olli-Pekka Rinta-Koski
	13 Variational Learning for Switching State-Space Models	Alexander Ilin
6.11.	14 Nonlinear Time-Series Prediction with Missing and Noisy Data	Karthikesh Raju
	10 Independent Factor Analysis	Ella Bingham

Additional material

Zoubin Ghahramani and Sam Roweis: Probabilistic Models for Unsupervised Learning, tutorial at 1999 Neural Information Processing Systems (NIPS'99).

Max Welling and Geoffrey E. Hinton. A New Learning Algorithm for Mean Feld Bolrzmann Machines. In Proceedings of the 12th International Conference on Artificial Neural Networks, pp.351-357, 2002.

Cecil Huang and Adnan Darwiche. Inference in Belief Networks: A Procedural Guide. International Journal of Approximate Reasoning, 15 no 3, pp 225-263, 1996.

Practical assignment

The practical assignment consists of two parts. In the first part, there are (simple) questions related to the topics covered during the course ps, pdf. In the second part you are asked to construct a program of your own ps, pdf. The exercises are to be completed before the 31st of January 2003. They should be handed in to the course assistant (room C311 in the computer science building).

More information

Additional information about the course may be asked from Jaakko Hollmén, and Salla Ruosaari.