The Multi-Layer Perceptron
A multi-layer perceptron is one of many different types of neural network. It comprises a number of active 'neurons' connected together to form a network. The 'strengths' or 'weights' of these links between the neurons is where the functionality of the network resides. Its basic structure is:
It is usually realised as electronic hardware or as computer software. Its original form though is biological: the whole idea of the neural network stems from studies of the structure and function of the human brain.
A neural network's usefulness is in its ability to mimic - or model - the behaviour of real-world phenomena like the weather, the environment, the economy, stock markets, population growth and decay, consumer buying habits, industrial processes, aircraft flight, limb movement.
Being able to model the behaviour of something, a neural network is able subsequently to classify the different aspects of that behaviour, recognise what is going on at the moment, diagnose whether this is correct or faulty, predict what it will do next, and if necessary respond to what it will do next.
Unlike mathematics, neural networks can faithfully model processes which are non-linear, whose internal workings can only be stated non-explicitly, and which can be only partially observed through many noise-ridden, non-independent variables. Which is what in fact all real-world processes are!
Instead of being programmed to model a real-world process explicitly, the neural network is trained to model it implicitly. Instead of modelling a real-world process in terms of logical and mathematical functions, the neural network models it as a vast array of 'inter-neural connection weights'.
A neuron is an active element which accepts input signals, multiplies each by a corresponding weight and applies the sum of the weighted inputs to a transfer function to produce an output signal:
A neurone's transfer function is usually a sigmoid function as shown above.
The design, testing and optimising of 'C' code for summing the weighted inputs is documented in WSUM.WRI. The compilable 'C' code embedded in an exerciser is in WSUM.C. The executable program is in WSUM.EXE. Similar treatment of the sigmoid function is in SIGMOID.WRI, SIGMOID.C and SIGMOID.EXE. The two pieces of code are then combined and tested as a complete neuron in NEURON.WRI, NEURON.C and NEURON.EXE.
Signals within the network are scaled to a range and precision which suits the hardware or software platform on which the network is implemented.
For example, a software implementation may run at its fastest when it is based on 16-bit integers. In this case, signals from the outside world would be rescaled to the range +32767 to -32767. Conversely, outputs from the network would be rescaled back to the range and precision expected by the outside world:
The vertical columns of neurons as shown in the following diagram are known as layers. A layer receives signals from the one before it and passes its outputs to the one which follows.
All networks have an input layer and an output layer. A network may also have hidden layers whose neurons have no direct connections to the outside world.
Below is a typical neural network showing the internal structure of a neuron:
We will implement this network as a 'C' function.
This page's parent within this Web Site. About this Web Site. Its home page. Email its Author.