In this video, we discuss the tradeoff between reconstruction and regularisation in VAEs. If the VAE is over-regularised, the reconstruction is affected as the encoded latent becomes independent of the data, and representations overlap to N(0,1). Now, this trade-off is managed by a weight on the regulariser, which leads to the beta-bae formulation. Next, to prevent over-regularisation and ensure the latents have sufficient information about the data, we also maximise the mutual information between the latent and data distributions. This leads to the info-vae formulation. Finally, we discuss the Python implementation of beta-VAE and show that it can be used for reconstruction, generation and dimensionality reduction.
Notes and Code: https://drive.google.com/drive/folders/1RscxoVDcDAKDFwaInYSHOdWCATpOsl-7
Intro Linear Algebra, Prob, ML: https://www.youtube.com/watch?v=XwVxx3GfRrE&list=PLcNLn_ApooUxkk-gOogvmUQen7nFWgdtg&index=1
