Deep Learning with Neural Networks- Part 2

Part 2: Feed-Forward Neural Network (FFNN)

This article, mainly discussing Feed Forward Neural Network(FFNN) which is commonly used in Supervised Learning with neural networks.

(1) In Supervised Learning Model

• Normally in the Supervised Learning model, there are mainly 4 types Feed Forward Neural Network(FFNN), Convolutional Neural Network (CNN), Recurrent Neural Network(RNN), and Encoder-Decoder Architectures.
• The main purpose of supervised learning neural networks is, give predictions by training with past data set for specific tasks. (Trainable)

(2) What is Feed Forward Neural Network?

• A Feed Forward Neural network is an artificial neural network in which the connections between nodes do not form a cycle.
• The opposite of a feed-forward neural network is a Recurrent Neural Network in which certain pathways are cycled.
• The feed-forward model is the simplest form of the neural network as information is only processed in one direction. While the data may pass through multiple hidden nodes, it always moves in one direction and never backward
• It is consists of 3 main layers, the input layer, an output layer, and hidden layers using weights and bias.

(3) How does a Feed-Forward Neural Network work?

• A Feed Forward Neural Network is commonly seen in its simplest form as a single layer perceptron.
• In this model, a series of inputs enter the layer and are multiplied by the weights. Each value is then added together to get a sum of the weighted input values.
• If the sum of the values is above a specific threshold, usually set at zero, the value produced is often 1, whereas if the sum falls below the threshold, the output value is -1.

(4) Single Layer perceptron concept

• The single-layer perceptron is an important model of feed-forward neural networks and is often used in classification tasks
• Furthermore, single-layer perceptrons can incorporate aspects of machine learning.
• Using a property known as the delta rule, the neural network can compare the outputs of its nodes with the intended values, thus allowing the network to adjust its weights through training in order to produce more accurate output values.
• This process of training and learning produces a form of gradient descent.

(5) Multi-Layer perceptron concept

• In multi-layered perceptrons, the process of updating weights is nearly analogous, however, the process is defined more specifically as back-propagation.
• In such cases, each hidden layer within the network is adjusted according to the output values produced by the final layer.

(5) Applications of Feed-Forward Neural network

• While Feed Forward Neural Networks are fairly straightforward, their simplified architecture can be used as an advantage in particular machine learning applications.
• For example, one may set up a series of feed-forward neural networks with the intention of running them independently from each other, but with a mild intermediary for moderation.
• Like the human brain, this process relies on many individual neurons in order to handle and process larger tasks. As the individual networks perform their tasks independently, the results can be combined at the end to produce a synthesized, and cohesive output.

e.g: pattern recognition, multivariate regression, robust regression, and handling of instrumental drifts.

This article is mainly about Feed Forward Neural Network(FFNN) which is commonly used in Supervised Learning with neural networks. The next article is mainly focusing on Convolutional Neural networks (CNN)