Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision | Qualitative Results

Qualitative results for the paper: Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision, 2019. Fredrik K. Gustafsson, Martin Danelljan, Thomas B. Schön. arXiv: https://arxiv.org/abs/1906.01620 Code: https://github.com/fregu856/evaluating_bdl Project page: http://www.fregu856.com/publication/evaluating_bdl/ We propose a comprehensive evaluation framework for scalable epistemic uncertainty estimation methods in deep learning. It is specifically designed to test the robustness required in real-world computer vision applications. We also apply our proposed framework to provide the first properly extensive and conclusive comparison of the two current state-of-the-art scalable methods: ensembling and MC-dropout. Our comparison demonstrates that ensembling consistently provides more reliable and practically useful uncertainty estimates. - All shown results are for ensembling with M = 8. - Street-scene semantic segmentation: 0:00 - 8:22. - - Cityscapes to Cityscapes (real to real): 0:00. - - Synscapes to Cityscapes (synthetic to real): 2:30. - - Synscapes to Synscapes (synthetic to synthetic): 5:00. - - Cityscapes to Synscapes (real to synthetic): 6:41 - Depth completion: 8:22 - 14:18. - - virtual KITTI to virtual KITTI (synthetic to synthetic): 8:22. - - virtual KITTI to KITTI (synthetic to real): 9:26. - On Cityscapes, the input image, prediction and predictive entropy are visualized. - On Synscapes, the input image, ground truth, prediction and predictive entropy are visualized. - For depth completion, the input image, input sparse depth map, ground truth depth map, prediction, predictive uncertainty, aleatoric uncertainty and epistemic uncertainty are visualized. - Black: minimum uncertainty, white: maximum uncertainty.