In this (short) homework, we'll implement Recurrent Neural Networks for character-level language modeling in Python.

We've provided the Linux kernel dataset (data/input.txt) but you're free to run experiments on a different dataset.

Download the starter code here.

Q1: Vanilla RNN (30 points)

Go through rnn.ipynb and implement the forward and backward pass for a 'vanilla' RNN. Code to check gradients and sample from the softmax distribution have already been provided. Implement beam search.

Deliverables

• Completed IPython notebook rnn.ipynb
  • Forward pass (10 points)
  • Backward pass (10 points)
  • Beam search (10 points)