Tik-61.261 Principles of Neural Computing
- Construct a MLP network which is able to separate the two classes
illustrated in Figure 1. Use two neurons both in the input and output
layer and an arbitrary number of hidden layer neurons. The output of
the network should be vector if the input vector belongs to
and if it belongs to class
. Use nonlinear activation functions, namely
McCulloch-Pitts model, for all the neurons and determine their weights
by hand without using any specific learning
- What is the minimum amount of
neurons in the hidden layer required for a perfect separation of the classes?
- What is the maximum amount of neurons in the hidden layer?
- The function
is approximated with a neural network. The activation
functions of all the neurons are linear functions of the input signals
and a constant bias term. The number neurons and the network
architecture can be chosen freely. The approximation performance of the network
is measured with the following error function:
is the input vector of the network and
- Construct a single-layer network which
minimizes the error function.
- Does the approximation
performance of the network improve if additional hidden layers are included?
- The MLP network of Figure 2 is trained for
classifying two-dimensional input vectors into two separate
classes. Draw the corresponding class boundaries in
the -plane assuming that the activation function
of the neurons is (a) sign, and (b) tanh.
- Show that (a)
is the solution of the
following difference equation:
where is a
positive momentum constant. (b) Justify the claims 1-3 made on the
effects of the momentum term in Haykin pp. 170-171.
- Consider the simple example of a network involving a single
weight, for which the cost function is
where , , and
are constants. A back-propagation algorithm with momentum is
used to minimize
. Explore the way in which the inclusion
of the momentum constant influences the learning process,
with particular reference to the number of epochs required for
convergence versus .
The MLP network.