Neuron is nothing but the central processing unit of the deep learning, In Neuron you might have multiple hidden layers which perform operations to give output to the next neuron or to the user.
Neuron is the basic building block of the neural networks, let me showcase some neuron in our body
In above Image, 1-9 are inputs to the neuron, 10-15 are output to the other neurons or to the user, 99(circle) is the central processing unit also know as neuron. In Neuron all the calculations occurs.
In Human, inputs are taking by eyes, ears, nose, skin... outputs are like hand legs, mouth so on..
I can understand you saying "enough with your biology bullshit", now jump to deep learning.
Neurons in Deep learning works same as neuron in biology, it takes inputs from the other neuron or from the user. Most of the time these inputs will be pre-processed, and all the calculations occurs in Neron as shown in below image.
I hope you can understand why everyone writes the Deep learning in below image format, In below image the Neuron can contain hidden layers of neurons Neuron also sometimes called the node and has input and output signals, input signals also called as synapses
Every input value is independent of other inputs, the input variable are for one single observation for example, a row in a excel sheet or in data base.
Every output is related to one observation(one row in database), the output can of below types:
Input layer accepts input features, It provides information from the data storage(xl, spreadsheet, database etc) to the network, no computation is performed at this layer, nodes here just pass on the information(features) to the hidden layer. In above image 1-9 are input layer also known as dentries
This is Neuron, Nodes of this layer are not exposed to the outer world, they are the part of the abstraction provided by any neural network. Hidden layer performs all sort of computation on the features entered through the input layer and transfer the result to the output layer.
Output layer shows up the information learned by the network to the outer world.
Activation function decides, whether a neuron should be activated or not by calculating weighted sum and further adding bias with it. The purpose of the activation function is to introduce non-linearity into the output of a neuron.
The output signal/synapses signal can be used as a input in the next layer of Neurons, or to the output.
A Neural Network without Activation function would simply be a Linear regression Model, which has limited power and does not performs good most of the times.
So most of the time, we would like out model to learn more than Linear Regression, so it is important to have activation function in Neural networks.
Also without activation function our Neural network would not be able to learn and model other complicated kinds of data such as images, videos , audio , speech etc.
Activation functions are important for a Artificial Neural Network to learn and understand the complex patterns.
The simplest form of activation function is binary — the neuron is either firing or not. The output y of this activation function is binary, depending on whether the input meets a specified threshold, θ.
The "signal" is sent, i.e. the output is set to one if the activation meets the threshold. A step function is a function like that used by the original Perceptron. The output is a certain value, A1, if the input sum is above a certain threshold and A0 if the input sum is below a certain threshold. The values used by the Perceptron were A1 = 1 and A0 = 0.
These kinds of step activation functions are useful for binary classification schemes. In other words, when we want to classify an input pattern into one of two groups, we can use a binary classifier with a step activation function. Another use for this would be to create a set of small feature identifiers.
Each identifier would be a small network that would output a 1 if a particular input feature is present, and a 0 otherwise. Combining multiple feature detectors into a single network would allow a very complicated clustering or classification problem to be solved.
A sigmoid function produces a curve with an "S" shape having upper limit as 1.0 and lower limit as 0.0
The sigmoid function takes any range real number and returns the output value which falls in the range of 0 to 1.
when ten value X is between -2 to 2 the curve of the Sigmoid becomes more steep
It's often used in logistic regression for a binary classification and also as activation function in neural networks.
The output of the sigmoid function can be interpreted as a probability. Sigmoid function can be used for many tasks in real life: classify junk mails, classify banks transactions, etc.
The output of the sigmoid function not only used to classify YES/NO(True/False), but also be used to determine the probability of YES/NO(True/False).
The Rectified Linear Unit is the most commonly used activation function in deep learning models. The function returns 0 if it receives any negative input, but for any positive value x it returns that value back.
So it can be written as
Graphically it looks like this
It's surprising that such a simple function (and one composed of two linear pieces) can allow your model to account for non-linearities and interactions so well. But the ReLU function works great in most applications, and it is very widely used as a result.
The Hyperbolic tangent (Tanh) function is an activation function which re-scales the values between -1 and 1 by applying a threshold just like a sigmoid function.
tanh is also one of the derivative from the sigmoid function, the values of a tanh is zero centered which helps the next neuron during propagating.
When we apply the weighted sum of the inputs in the tanh(x), it re scales the values between -1 and 1. The large negative numbers are scaled towards -1 and large positive numbers are scaled towards 1.
If you zoom in the above images, you can understand that as the x value increase(in positive and negative side), the curve tends to become vertical.
Myself KarthiQ, I am the author of this blog, I know ways to write a good article but some how I donot have the skills to make it to reach people, would you like help me to reach more people By sharing this Article in the social media.