Lecture 7 - Transformers for NLP & LLMs
Teacher: Eleni Metheniti (ANITI)
Contents
- Self-attention
- The Transformer architecture
- (Large) Language Models
- Transformers for Embedding : BERT
- Transformers for Generation : GPT2
- Application : Finetuning for multi-class classification and Text generation with Transformers
Prerequisites:
Knowledge about neural networks principles; knowledge of several NN layer types; gradient descent & backpropagation; basics of NLP; RNN principles
Slides
Download the slides here
Notebook
- Colab notebook for lecture here
- Colab notebook for homework here
- Curated notebooks on LLM applications here
Further reading:
See final slide