Deep Learning with Neural Networks- Part 4

Part 4: Recurrent Neural Networks(RNN)

This article, mainly discussing the Recurrent Neural Network(RNN) which is one of the main part of Supervised Learning with neural networks working with time series data.

(1) What are Recurrent Neural Networks?

  • A recurrent neural network (RNN) is a type of artificial neural network which uses sequential data or time series data.

(2) What are the applications based on Recurrent Neural Networks?

Recurrent Neural Networks are mainly working with time series data.

  • Speech Recogintion

(3) What are the types of Recurrent Neural Network

  • Like feedforward and convolutional neural networks (CNNs), recurrent neural networks utilize training data to learn

(4) How Recurrent Neural Network works

  • RNN converts the independent activations into dependent activations by providing the same weights and biases to all the layers, thus reducing the complexity of increasing parameters and memorizing each previous outputs by giving each output as input to the next hidden layer.

(5) Advantages of Recurrent Neural Network

  • An RNN remembers each and every information through time. It is useful in time series prediction only because of the feature to remember previous inputs as well. This is called Long Short Term Memory.

(6) Main issues of standard RNN

  • There are two major obstacles RNN’s have had to deal with gradient issues.

(a) Exploding Gradient issue- Assigning high importance to the weights.This problem can be easily solved by truncating or squashing the gradients.

(b) Vanishing Gradient issue- Vanishing gradients occur when the values of a gradient are too small and the model stops learning or takes way too long as a result.Fortunately, it was solved through the concept of LSTM

(7) Varients of RNN Architectures

  • To solving the problems occured in RNN, found some varients of RNN architectures.

(a) Bidirectional recurrent neural networks (BRNN): These are a variant network architecture of RNNs.Bidirectional RNNs pull in future data to improve the accuracy of it.

(b) Long short-term memory (LSTM): This is a popular RNN architecture, as a solution to vanishing gradient problem.LSTMs have “cells” in the hidden layers of the neural network, which have three gates–an input gate, an output gate, and a forget gate. These gates control the flow of information which is needed to predict the output in the network.

( c) Gated recurrent units (GRUs): This RNN variant is similar the LSTMs as it also works to address the short-term memory problem of RNN models. Instead of using a “cell state” regulate information, it uses hidden states, and instead of three gates, it has two — a reset gate and an update gate.

This article is mainly about Recurrent Neural networks (RNN)which are commonly used in Supervised Learning with neural networks. For the the problems in RNN, LSTM, GRUs , it has founded Tranformer model artictecture as for the solutions. It gives accurate outputs to the RNN based problems. In next articles we will talk about that.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store