Deep recurrent neural networks¶
Most RNN consists of 3 blocks of parameters.
From input to hidden
From previous hidden state to the next hidden state
From hidden state to the output
There are multiple options how to implement deep RNN.
we can have a hierarchy of hidden states
we can add an extra step between hidden states
we can have skip connection between hidden states