Laboratory of Computer and Information Science / Neural Networks Research Centre CIS Lab Helsinki University of Technology
[ Main Page ]

Some resources for matrix derivation and Normal distributions:


Lecture 1: NFL Theorems are discussed in The Lack of A Priori Distinctions Between Learning Algorithms, David H. Wolpert, Neural Computation 8, 1341-1390 (1996), sections 1-4.

Lecture 3: A light article on Bayes is The Economist. London: Jan 7, 2006. Vol. 378, Iss. 8459; p. 73. On a machine in, go to The Economist

The Dutch Book Argument shows that priors must be probabilities unless one accepts irrational decisions. More details are in Notes by D. Freedman

More on Cox's Axioms can be found in K. S. Van Horn, Constructing a logic of plausible inference: a guide to Cox's Theorem, International Journal of Approximate Reasoning 34, no. 1 (Sept. 2003), pp. 3-24. Citeseer

Lecture 4: The demo illustrating overfitting avoidance is based on a generalized linear model with Normal likelihood and a Normal prior. Details can be found in C.K.I. Williams, Prediction with Gaussian Processes, Learning in Graphical Models, ed. Michael I. Jordan, MIT Press, 1999.

Lecture 6: Some background on the example illustrating a nonuniform prior for scale data: Benford's Law

Lecture 8: Simulation methods are a very large and active topic: the lecture only introduced the basic ideas. Introductions to MCMC include

Knowing when a simulation has converged is a major difficulty in simulation methods. Perfect Sampling offers an interesting way of overcoming this issue in certain type of situations.

Lecture 9: An example of applying variational approximation is found at Fergus et al., Removing Camera Shake From a Single Image

Lecture 10: Some resources on the EM algorithm:

Lecture 11: A standard missing data reference is Statistical Analysis with Missing Data, Little and Rubin, 2nd ed., Wiley 2002.
Data augmentation was proposed in The Calculation of Posterior Distributions by Data Augmentation, M. Tanner and W. Wong, Journal of American Statistical Association, vol. 82, no. 398, pp. 528-540, 1987.

Lecture 12: MacKay's book explains Gaussian Processes in chapter 45. Ways of solving classification problems using GP:s:

Lecture 13: Making Hard Decisions: an Introduction to Decision Analysis, Robert T. Clemen, Duxbury Press, 1995, is a gentle introduction to decision analysis.

For an example where costs do matter in model selection, see Stochastic optimization methods for cost-effective quality assessment in health, D. Fouskakis and D. Draper, 2005. (submitted article)

Model selection is discussed in David MacKay's book in chapter 28. Especially the use of evidence and the resulting preference for simpler models is explained.

[ Main Page ]

You are at: CIS → T-61.5040 /

Page maintained by t615040 (at), last updated Wednesday, 25-Apr-2007 14:03:15 EEST