Tik-61.261 Principles of Neural Computing
Raivio, Venna

Exercise 2
  1. The error-correction learning rule may be implemented by using inhibition to subtract the desired response (target value) from the output, and then applying the anti-Hebbian rule. Discuss this interpretation of error-correction learning.

  2. Figure 1 shows a two-dimensional set of data points. Part of the data points belongs to class $ C_1$ and the other part belongs to class $ C_2$. Construct the decision boundary produced by the nearest neighbor rule applied to this data sample.

  3. A generalized form of Hebb's rule is described by the relation

    $\displaystyle \Delta w_{kj}(n)=\alpha F(y_{k}(n))G(x_j(n))-\beta w_{kj}(n)F(y_k(n))$    

    where $ x_j(n)$ and $ y_k(n)$ are the presynaptic and postsynaptic signals, respectively; $ F(\cdot)$ and $ G(\cdot)$ are functions of their respective arguments; and $ \Delta w_{kj}(n)$ is the change produced in the synaptic weight $ w_{kj}$ at time $ n$ in response to the signals $ x_j(n)$ and $ y_j(n)$. Find the balance point and the maximum depression that are defined by this rule.

  4. An input signal of unit amplitude is applied repeatedly to a synaptic connection whose initial value is also unity. Calculate the variation in the synaptic weight with time using the following rules:
    1. The simple form of Hebb's rule described by

      $\displaystyle \Delta w_{kj}(n)=\eta y_k(n) x_j(n)$    

      assuming the learning rate $ \eta=0.1$.
    2. The covariance rule described by

      $\displaystyle \Delta w_{kj}=\eta (x_j-\overline{x})(y_k-\overline{y})$    

      assuming that the time-averaged values of the presynaptic signal and postsynaptic signal are $ \overline{x}=0$ and $ \overline{y}=1.0$, respectively.

  5. Formulate the expression for the output $ y_j$ of neuron $ j$ in the network of Figure 2. You may use the following notations:

      $\displaystyle x_i =$   ith input signal    
      $\displaystyle w_{ji} =$   synaptic weight from input i to neuron j    
      $\displaystyle c_{kj}=$   weight of lateral connection from neuron k to neuron j    
      $\displaystyle v_j =$   induced local field of neuron j    
      $\displaystyle y_j =\varphi(v_j)$    

    What is the condition that would have to be satisfied for neuron $ j$ to be the winning neuron?

    Figure 1: Data point belonging to class $ C_1$ and $ C_2$ are plotted with 'x' and '*', respectively.
    \begin{figure}\centering\epsfig{file=ex2_2.ps,width=130mm}\end{figure}

    Figure 2: Simple competitive learning network with feedforward connections from the source nodes to the neurons, and lateral connections among the neurons.
    \begin{figure}\centering\epsfig{file=ex2_5.eps, width=40mm }\end{figure}



Jarkko Venna 2005-04-13