Highway Networks - Deep Neural Network Explained

👨‍💻 to get started with AI engineering, check out this Scrimba course: https://scrimba.com/the-ai-engineer-path-c02v?via=yacineMahdid Highway Networks are a type of network inspired by LSTM that make use of learnable information highway to let inputs flow unimpeded to subsequent layers. They have a lot of similarities with residual neural networks and offer deep insight into how to make training deeper neural networks possible. # Table of Content - Introduction: 0:00 - Degradation Problem: 0:38 - Idea Behind Highway Networks: 1:14 - Formulas: 2:11 - Training & Data: 2:57 - Plain VS Highway: 3:34 - MNIST Sanity Checks: 4:19 - FitNet vs Highway: 4:37 - SOTA vs Highway: 5:00 - Highway Activation Analysis: 5:26 - Highway Ablation Analysis: 7:34 - Conclusion: 8:39 Highway Networks Paper: https://arxiv.org/pdf/1505.00387v2 Training Very Deep Network Paper: https://arxiv.org/pdf/1507.06228 For an implementation of Highway Networks do check this repository: https://github.com/protonx-tf-03-projects/highway-networks?tab=readme-ov-file ---- Join the Discord for general discussion: https://discord.gg/QpkxRbQBpf ---- Follow Me Online Here: Twitter: https://twitter.com/yacineaxya GitHub: https://github.com/yacineMahdid LinkedIn: https://www.linkedin.com/in/yacinemahdid/ ___ Have a great week! 👋