Neural Networks

The idea of neural networks is based upon the design of the human brain: if we are trying to build artificial intelligence to simulate the behavior of a human brain, then the design of that AI should mimic the human brain. In particular, if the brain consists of a (large) network of connected neurons, then an AI system should also use a set of artificial neurons. Just as the brain consists of neurons which either fire, or don't fire, in any given situation, and the firing or non-firing of those neurons influence whether other, connected neurons, fire or not, an artificial brain-a neural network-consists of a set of neurons which either fire or don't fire in any given situation.

A neural network (usually just called a neural net) generally consists of a set of input neurons, a set of output neurons, and one or more "hidden layers" of neurons. The inputs are not directly connected to the outputs but rather are indirectly connected through the hidden layers. The precise manner in which the hidden layers are designed-the parameters under which they operate-is something over which the designer of the net usually has control. A typical neural net with two hidden layers might look something like this:

Neural nets are generally trained using a large number of training instances. That is, usually we know in advance what the correct output is for a large number of sets of inputs. We then train the network so that the output for those known inputs matches, as closely as possible, the correct output. Once the network has been trained in this fashion, we then have reasonable confidence that when presented with an input it has never seen before, it will produce an output which is correct or is quite close to being correct.

It is important to realize that the design of a neural net is not an exact science-in this way it differs from, say, a rule-based system where the design of each rule is quite precise and accurate. Just as it is usually impossible to predict, with perfect accuracy, how a human will act in any given situation, it is impossible also to predict how a neural net will behave in any given situation. As such, neural nets tend to be used in forecasting types of situations where perfect accuracy is neither necessary nor possible, but making an intelligent prediction carries with it significant benefits as opposed to just doing nothing. They are often used in fraud detection and in financial forecasting (e.g. predicting the performance of Wall Street stock indices).

Neural nets-sometimes known as "connectionism"-have been around in one form or another since the 1960's. The biggest potential error in designing neural nets has always been one of overtraining it. If the people designing the net are allowed access to all known training instances, then they will generally be able to tune the net so that it performs extremely well over all those instances. When it goes into production, however, it fails miserably because it is seeing, for the first time, data for which it has not been trained. What is needed, instead, is a large number of training instances, and to rigorously control access to training instances so that the developers are allowed access only to a small number of training instances. When the net is being tested, by people other than the original developers, a different set of training instances-one which the developers have never seen-is then used to test performance.

In other, what is needed is a set of rigorous quality control standards. The challenge has often been, however, that neural nets are often designed in somewhat loosely structured R&D labs where such standards have been lacking, and so unfortunately performance, and hence the reputation of the technology, needlessly suffers.

Another challenge with neural nets is that most computers on which they are implemented have only one or a relatively small number of processors and so allow for only a limited amount of parallelism. The human brain is an example of a massively parallel architecture. Any of a very large number of neurons in the brain can fire at the same time, and one neuron doesn't have to wait for billions of other neurons to decide if they will fire before it makes its own decision. Unless a neural net is implemented on a similarly massively parallel architecture, its ability to really simulate the behavior of the human brain is going to be limited. Such architectures are certainly the subject of research in academia and at corporate R&D labs like at Sun Microsystems, but they are not really available to the average computer user at present.

Further information about neural networks may be found in
Neural Networks in Finance and Investing by Trippi and Turban.



Next week: AI Languages



Home: Ramalila.NET



All copyrights are maintained by respective contributors and may not be reused without permission. Graphics and scripts may not be directly linked to. Site assets copyright © 2000 RamaLila.com and respective authors.
By using this site, you agree to relinquish all liabilities and claims financial or otherwise against RamaLila and its contributors. Visit this site at your own risk.