Tik-61.261 Principles of Neural Computing
Raivio, Venna

Exercise 10

1. The thin-plate-spline function is described by

   $$\varphi(r)=\left(\frac{r}{\sigma}\right)^2\log\left(\frac{r}{\sigma}\right) \sigma>0 r \in \mathbb{R}^{+}$$

   
   
   Justify the use of this function as a translationally and rotationally invariant Green's function. Plot the function graphically. (Haykin, Problem 5.1)

   
   

2. The set of values given in Section 5.8 for the weight vector $\mathbf{w}$ of the RBF network in Figure 5.6. presents one possible solution for the XOR problem. Solve the same problem by setting the centers of the radial-basis functions to

   $$\mathbf{t}_1=[-1,1]^T \mathbf{t}_2=[1,-1]^T.$$

   
   
   (Haykin, Problem 5.2)

   
   

3. Consider the cost functional

   $$\mathcal{E}(F^{\ast}) = \sum_{i=1}^{N} \left[d_i-\sum_{j=1}^{m_1}... ...bf{x}_j -\mathbf{t}_i \Vert)\right]^2 + \lambda \Vert\mathbf{D}F^{\ast} \Vert^2$$

   
   
   which refers to the approximating function

   $$F^{\ast}(\mathbf{x})=\sum_{i=1}^{m_1} w_i G(\Vert \mathbf{x}- \mathbf{t}_i \Vert).$$

   
   
   Show that the cost functional $\mathcal{E}(F^{\ast})$ is minimized when

   $$(\mathbf{G}^T\mathbf{G}+\lambda \mathbf{G}_0)\mathbf{w}=\mathbf{G}^T\mathbf{d}$$

   
   
   where the $N$-by-$m_1$ matrix $\mathbf{G}$, the $m_1$-by-$m_1$ matrix $\mathbf{G}_0$, the $m_1$-by-1 vector $\mathbf{w}$, and the $N$-by-1 vector $\mathbf{d}$ are defined by Equations (5.72), (5.75), (5.73), and (5.46), respectively. (Haykin, Problem 5.5)

   
   

4. Consider more closely the properties of the singular-value decomposition (SVD) discussed very briefly in Haykin, p. 300.
   1. Express the matrix $\mathbf{G}$ in terms of its singular values and vectors.
   2. Show that the pseudoinverse $\mathbf{G}^{+}$ of $\mathbf{G}$ can be computed from Equation (5.152):

      $$\mathbf{G}^{+}=\mathbf{V}\mathbf{\Sigma}^{+}\mathbf{U}^T.$$

      
      

   3. Show that the left and right singular vector $\mathbf{u}_i$ and $\mathbf{v}_j$ are obtained as eigenvectors of the matrices $\mathbf{GG}^T$ and $\mathbf{G}^T\mathbf{G}$, respectively, and the squared singular values are the corresponding nonzero eigenvalues.