Neural Network (Learning Algorithms), The pattern recognitioning usage of AI

[DaRock]

Neural Networks

"Simulate Networks within Code to create Learning or Pattern Recognition Systems"

Overview
A neural network is a way of trying to simulate the brain electronically. So to understand how a neural net works we first must have a look at how brain works.

Our brains are made up of about 100 billion tiny units called neurons. Each neuron is connected to thousands of other neurons and communicates with them via electrochemical signals. Signals coming into the neuron are received via junctions called synapses, these in turn are located at the end of branches of the neuron cell called dendrites. The neuron continuously receives signals from these inputs and then performs a little bit of magic. What the neuron does (this is over simplified I might add) is sum up the inputs to itself in some way and then, if the end result is greater than some threshold value, the neuron fires. It generates a voltage and outputs a signal along something called an axon.


The Math
A neuron can have any number of inputs from one to n, where n is the total number of inputs. The inputs may be represented therefore as x1, x2, x3… xn. And the corresponding weights for the inputs as w1, w2, w3… wn. Now, the summation of the weights multiplied by the inputs we talked about above can be written as x1w1 + x2w2 + x3w3 …. + xnwn, which I hope you remember is the activation value. So…

a = x1w1+x2w2+x3w3... +xnwn

Remember that if the activation > threshold we output a 1 and if activation < threshold we output a 0.


Using a neuron
Well, we have to link several of these neurons up in some way. One way of doing this is by organising the neurons into a design called a feedforward network. It gets its name from the way the neurons in each layer feed their output forward to the next layer until we get the final output from the neural network.

Each input is sent to every neuron in the hidden layer and then each hidden layer’s neuron’s output is connected to every neuron in the next layer. There can be any number of hidden layers within a feedforward network but one is usually enough to suffice for most problems you will tackle. There can be any number of neurons in each layer, it all depends on the problem.

Once the neural network has been created it needs to be trained. One way of doing this is initialize the neural net with random weights and then feed it a series of inputs which represent the different panel configurations. For each configuration we check to see what its output is and adjust the weights accordingly so that whenever it sees something looking like a number 4 it outputs a 1 and for everything else it outputs a zero. This type of training is called supervised learning and the data we feed it is called a training set. There are many different ways of adjusting the weights, the most common for this type of problem is called backpropagation

How it works
Basically, in short the algorithm requires a learning "curve" which in this case is a wave form similar to a SIN wave or a COS wave which moves itself around according to vectors that it sets where it determines and edge to a shape in a a image; this can be anything from a path in a picture to face recognition. It over a series of tries attempts to recreate that path, and remember it.

You could then theoritcally take a series of images and create a "future" angle to understand where a target was headed, and if it was probable to change course and which way it was going to go. This is a true learning alogrithm and the basis for all real Artifical Intelligence.



Hope this was informative,

-DaRock

[The Pentium Guy]

How does sin/cos get involced in AI? Holy shit! I understand what you're saying... but man wow. That's incredible. I just never imagined it that way.

I should start learning basic AI first .

-The Pentium Guy

[AmdDarkTech]

AI >_< drives me nuts!

[The Pentium Guy]

We think on the same plane huh? Don't worry, I'll get it one day

[DaRock]

Let me try to simplify the idea so the developers can wrap an understanding around it.

All a Neural Net actually does is Map a series of Inputs to an Output to get a best guess or approximation value.

Now we can do that now if we wanted to, and most developers are thinking why would I want to do that? Well because in things such as Hand Writting not everyone writes the letter "A" or "a" the same way there are many different variations however we can use an approximation to understanding the image to receive a result; which standard mathematics and programming cannot do without extensive logic.

The advantage to this type of AI is that it can do pattern recognition as an inhiernt portion of the Net via Learning Algorithms; now, for clarification it is important to note that Neural Net(s) are seperate identites or "classes", the Neural Net invokes a learning algorithm to enhance it's understanding. However the Neural Net does do something realtively unique.

It can run functions in Parrallel; which as coders might be hard to understand. Meaning it can extract one or more value within one function and set a series of other Neural Nets off to begin other learning processes on something else in which the Neural Net has determined as "realtive" to the current process.

The Structure of a Neural Net is a "Input Layer" which is a series of Inputs (>=1) representing a value (decimal or float); which are connected to a "Hidden layer", at least 1 Hidden Layer there may be more interconnected Hidden Layers, each layer containing a specific "Type" of Function which produces a unique value according to the Input; they are Interconnected by Output(Hidden Layer1) to Input(HiddenLayer2); the output of each Hidden Layer is called the "Output Layer" however Hidden Layers which are interconnected do not report the internal Output Layers only the last Output Layer does; this is the "Output Layer" for the Neural Net.

Inside the Input layer are things called Neurons; which I will describe the structure of shortly; these Neurons are then interconnected to a type of function as the input within the Hidden layers; these function types are pretty standard, Sigmoid, Linear, Hard-track etc.. Basically the represent either a wave function, a linear straight line, or a series of broken horizontal lines; there are many different function types I am only labeling these 3 for demostration purposes.

These Functions after execution then compare to a Bias; or offset; to determine the Output value of X? If X <=1 then Value = 0 else Value = 1; on or off, basically, was enough information sent to the neural net for the machine to remember that function or not? If there was flip the switch otherwise do nothing.


To get an idea; the Neural Networking algorythms (at least some of them) are modelled after the brain (not necessarily - human brain) and how it processes the information. The brain is a very efficient tool. Having about 100,000 times slover response time than computer chips, it (so far) beats the computer in complex tasks, such as image and sound recognition, motion control and so on. It is also about 10,000,000,000 times more efficient than the computer chip in terms of energy consumption per operation.

The brain is a multi layer structure (think 6-7 layers of neurons) that works as a parallel computer capable of learning from the "feedback" it receives from the world and changing its design (think of the computer hardware changing while performing the task) by growing new neural links between neurons or altering activities of existing ones.


Now you can interconnect Neural Nets; to other Neural Nets; which can then do things as; My Bot in the game is being hit by a sword it takes 20% damage; the Neural Net fires saying last time at 20% I healed myself; it fires another Neural net saying if I am with in X radius I cannot heal; so Run away. However it remembers that it didn't do the first action and triggers the net again then Healing the in game AI Bot.

So now; the Neural Net demostraits something else; it's learning capacity; it will learn the environment and the players playing style. So forget about the Pong Logic.

Now there are several already established Learning Algorithms; Feedforward Backpropagation, Genetic, etc.. You need to determine which is more correct.

Basically this is programming; except you are now programming off of "eh it's close enough..." type attitude rather than it needs to be exactly 4.5434121 degrees to the right; instead the Neural Net will say 4.547 is close enough to match the pattern if that makes sense.

Neurons are the smallest unit in a Neural Net they hold basically information being sent; an input an output; error threshold; and value threshold. These are then Mapped via an array to an input on the hidden layer; each input on the hidden layer is connected to each neurons output or value; it then sums the difference up and determines a new value which is then used within the Hidden Layers Activation Function.

----------------------
Simple Neural Net
----------------------
[Input] [ Hidden] [Output]
W1 0---\
W2 0-------[f(a)-to-(LA)--->
W3 0__/ /
.........../
......... /
Wz 0_/

Wx=Weights 1,2,3, etc... z
LA = Learning Algorithm

Hopes the helps,

-DaRock

[elnerdo]

Whoa..... That's awesome..

[The Pentium Guy]

Very. I have yet to start basic AI.

[Loffen]

yay.. i've alwyas been facinated by AI, yet, shurdlu (or shurdul..) is the most advanced thingy i've seen (and it doesnt seem to be very advanced)...

-- Loffen

[The Pentium Guy]

On the topic of Neural Network, my uncle called some sort of international conference a few years ago regarding how Neural Networks (simulating the brain through electronics) are the future of computing.

I read through the book which he, and a bunch of other authors, wrote.....pretty damn interesting (of course I got confused, but man! interesting developments are going on). I should start to take this stuff seriously now.

-The Pentium Guy

[Loffen]

Garr... impossible to find stuff about em.. all i can find are those "cheap" nets, which only are able to learn one thing at a time..

-- Loffen

[elnerdo]

I tried for a little bit. I got to the point of making a neural net that could compute AND and OR.

[Loffen]

Heh.. cool.. wanna teach me? would be so cool

-- Loffen