Tik-61.261 Principles of Neural Computing
Raivio, Venna

Exercise 10
  1. The thin-plate-spline function is described by

    $\displaystyle \varphi(r)=\left(\frac{r}{\sigma}\right)^2\log\left(\frac{r}{\sigma}\right)$      for some  $\displaystyle \sigma>0$      and  $\displaystyle r \in \mathbb{R}^{+}$    

    Justify the use of this function as a translationally and rotationally invariant Green's function. Plot the function graphically. (Haykin, Problem 5.1)

  2. The set of values given in Section 5.8 for the weight vector $ \mathbf{w}$ of the RBF network in Figure 5.6. presents one possible solution for the XOR problem. Solve the same problem by setting the centers of the radial-basis functions to

    $\displaystyle \mathbf{t}_1=[-1,1]^T$      and  $\displaystyle \mathbf{t}_2=[1,-1]^T.$    

    (Haykin, Problem 5.2)

  3. Consider the cost functional

    $\displaystyle \mathcal{E}(F^{\ast}) = \sum_{i=1}^{N} \left[d_i-\sum_{j=1}^{m_1}...
...bf{x}_j -\mathbf{t}_i \Vert)\right]^2 + \lambda \Vert\mathbf{D}F^{\ast} \Vert^2$    

    which refers to the approximating function

    $\displaystyle F^{\ast}(\mathbf{x})=\sum_{i=1}^{m_1} w_i G(\Vert \mathbf{x}- \mathbf{t}_i \Vert).$    

    Show that the cost functional $ \mathcal{E}(F^{\ast})$ is minimized when

    $\displaystyle (\mathbf{G}^T\mathbf{G}+\lambda \mathbf{G}_0)\mathbf{w}=\mathbf{G}^T\mathbf{d}$    

    where the $ N$-by-$ m_1$ matrix $ \mathbf{G}$, the $ m_1$-by-$ m_1$ matrix $ \mathbf{G}_0$, the $ m_1$-by-1 vector $ \mathbf{w}$, and the $ N$-by-1 vector $ \mathbf{d}$ are defined by Equations (5.72), (5.75), (5.73), and (5.46), respectively. (Haykin, Problem 5.5)

  4. Consider more closely the properties of the singular-value decomposition (SVD) discussed very briefly in Haykin, p. 300.
    1. Express the matrix $ \mathbf{G}$ in terms of its singular values and vectors.
    2. Show that the pseudoinverse $ \mathbf{G}^{+}$ of $ \mathbf{G}$ can be computed from Equation (5.152):

      $\displaystyle \mathbf{G}^{+}=\mathbf{V}\mathbf{\Sigma}^{+}\mathbf{U}^T.$    

    3. Show that the left and right singular vector $ \mathbf{u}_i$ and $ \mathbf{v}_j$ are obtained as eigenvectors of the matrices $ \mathbf{GG}^T$ and $ \mathbf{G}^T\mathbf{G}$, respectively, and the squared singular values are the corresponding nonzero eigenvalues.

Jarkko Venna 2005-04-13