Courses in previous years: [ 2001 | 2002 | 2003 | 2004 ]

Tik-122.102 Special Course in Information Science V L

Regularization and sparse approximations

Regularization and sparse approximations

Course description

Part I: Regularization and sparse basis function approximations

Regularization is a technique to constrain the complexity of a regression mapping by extending the conventional error function (for instance, the squared error between the target values and model outputs) by an additional penalty term that adds a penalty for parameter values that are not desired, e.g. mappings that are wavy (high curvature).

Regularization techniques will be covered from the classical ridge regression techniques to the more recent sparse regression algorithms. Function approximation will be approached using a linear superposition of basis functions. Sparsity of the linear weights in the superposition will be a criterion in learning from data. Variations on this theme will be covered in the light of scientific journal articles.

Part II: Support vector machines

The second part provides insights to the sparse approximations from the viewpoint of support vector machines.

Course material

Selected scientific articles, textbooks and lectures will form the materials for the course.

Homework 1

Exercise paper: exercise1.pdf
Data for the first homework: polydata.dat
The first column of the data matrix polydata is a variable x and the second column is a variable y.

Project work of the part I

Exercise paper: projectwork.pdf
Data for the project work: Systemdata.dat

Timetable

The lecture topics can be found in the timetable