Lecture 7 - Recurrent Neural Networks for text processing
Teacher: Romain Bielawski (ANITI)
Very important: due to a strong response in the community, we are splitting the class in 2 sessions. This page is for the advanced session, appropriate for students with prior training in Maths, Engineering and/or Computer Science. If you are here by mistake, go to [the main/basic session page].
Lecture video
View the recorded lecture here (this will only be available for approximately 6 weeks after the course)
Contents
- Sequential data and variable size inputs
- Recurrent neural network principles, hidden states
- Backprop through time
- Vanishing gradient
- RNN, LSTM, GRU
- Sentence embeddings, text generation, seq2seq, machine translation
- Application : LSTM for text generation & text classification
Slides
Download the slides here
Notebook
Access the collab notebook here
Prerequisites:
Knowledge about neural networks principles; knowledge of several NN layer types; gradient descent & backpropagation; basics of NLP