Week 13: Attention and CNN

Week 13: Attention and CNN

Announcements

Class participation, EdSTEM and Google Folder.
(Back to Course Index Page)

Tuesday

Attention and Transformer

Attention is all you need, that paper
Textbooks
Courses slides
Courses videos

Example Paper

Attention? Attention Blog by Lilian Weng
The Illustrated Transformer Blog by Jay Alammar
ViT: Transformers for Image Recognition Paper Blog Video
ViT on EEG data paper code

Example Code

Hugging Face’s Transformers library
TensorFlow and PyTorch official tutorials
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Google Colab

Thursday

Convolutional Neural Network (CNN)

Backpropagation
Course video sample 01
Backpropagation review Slides
Backpropagation review Colab
Calculus on Computational Graphs: Backpropagation link
How the backpropagation algorithm works, link
Convolutional Neural Network (CNN)

Final Project

Last Draft due this weekend. (Lab 9 Due Sunday, 12/03/2023, by midnight (23:59, EST))
Last Peer review due next Tuesday. (Lab 10 Due Tuesday, 12/05/2023, before class (6:09, EST))
Final Project due Tuesday, 12/12. (Final Project Due Tuesday, December 12th, by midnight (23:59 EST))