Neural Networks : Brains from Circuits

Like Fuzzy Logic, neural networks are designed to deal with noisy, imprecise values by simulating a parallel structure. They consist of heavily interconnected small pieces of code, most of which have only one output. Neural networks are man's attempt to recreate the computing potential of the brain. To date however, try as they might, no one has rightfully claimed the title of being able to simulate anything as complex as the brain. Part of the problem is that the human brain has billions of neurons, whereas even the most advanced Artificial Neural Networks, or ANNs, have at most 1000 artificial neurons.

Neural networks imitate actual neurons, which have many different inputs and one output. They analyze these inputs, and only output (or fire) when the correct sequence is met. This system is called a firing rule, and it allows neural networks to have greater flexibility. This also allows for easy pattern recognition when dealing with complicated input/output patterns. A neural network, unlike other systems, can also “weigh” inputs, only firing when appropriate.

A Manufactured Minds visual representation of the firing rule.

Scientists have realized for some time that they need to understand connection strength between the neurons in our brains before trying to improve upon the mechanics of Artificial Neural Networks. Connection strength is defined as the ability of a neuron to connect to and influence its surrounding neurons. Different day-to-day activities, such as reading or writing, can have an astounding effect on our neural connective strength. With use, the connective strength becomes stronger. With lack of use, our neural connective strength weakens and may even disappear altogether.

A Manufactured Minds visual representation of a) a strong connective strength between neurons, and b) a weak connective strength between neurons.

Another important concept being researched is the inhibition/excitation distinction. Excitation is when a neuron has enough information or stimulus to fire; inhibition is when it doesn’t. This distinction relies on the connection strength between two connected neurons. One final concept worth mentioning is the transfer function. The transfer function compares the neuron’s firing rate with how much input it receives from the other connected neurons. This allows it to fire at the correct frequency and with the correct information. The more input from neurons, the stronger the firing rate.

A Manufactured Minds visual representation of a) exhibition, and b) inhibition.

Depending on their structure, Artificial Neural Networks are more organized than Biological Neural Networks. This organization is based upon three things – input nodes, ‘hidden’ nodes, and output nodes. Input nodes take in the information. As the information is taken in and sent across the neuron, each node is given an activation number, depending on the importance of the information supplied. The higher the activation number, the more activity that is taking place as information is being passed through the nodes. Artificial neurons indicate activity by passing the activation number through its nodes, whereas biological neurons convey their activation number by firing more frequently.

A Manufactured Minds visual representation of a neural network model containing hidden nodes (represented here in orange and yellow).

Actual neurons in our brain may have as many as 10000 different inputs from different neurons. Most neural networks, however, even though they can contain hundreds of these inputs, are not nearly as complicated.

There are two types of neural networks: feed-forward and feed-back networks. A feed-forward network can only move information forward, from input to output, without any backtracking along the way. This is useful with pattern recognition.

A feed-back network travels in all directions, allowing its output to be used in previously used “neurons”. This setup is often used to solve more complicated problems and can be researched further if needed.

A Manufactured Minds visual representation of a) a feed-forward neural network and b) a feed-back neural network.

See also: Fuzzy Math, Expert systems