Vision Transformer Attention from “Emerging Properties in Self-Supervised Vision Transformers” paper explained as fast as possible.
An episode of AIQuickie with PyTorch1.x code.
▬ Contents of this video ▬▬▬▬▬▬▬▬▬▬ 👀
0:00 - Intro
0:30 - Theory part
4:39 - Code part
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Link to the Notebook: https://github.com/EscVM/EscVM_YT/blob/master/Notebooks/2%20-%20PT1.X%20DeepAI-Quickie/pt_1_vit_attention.ipynb
Link to the paper: https://arxiv.org/abs/2104.14294
