Event-based Vision Algorithms, Kostas Daniilidis

Asynchronous event-based sensors allow us to capture the environment fast, under low light or high dynamic range conditions, and without motion blur. However, traditional frame-based vision approaches cannot be applied due to the asynchronous nature of incoming measurements. We introduce novel approaches for feature tracking, optical flow, and stereo using purely events as inputs. Optical flow is learnt with a neural autoencoder network that uses as input at every pixel only the trail of timestamps. We present generative networks for simulating events that enable the use of annotated classic camera datasets for training. Last but not least, we have built one of the most extensive event datasets (MVSEG) covering day and night, city and indoor environments from a camera setup mounted on a drone, a motorcycle, and a car. Kostas Daniilidis is the Ruth Yalom Stone Professor of Computer and Information Science at the University of Pennsylvania where he has been faculty since 1998. He is an IEEE Fellow. He was the director of the GRASP laboratory from 2008 to 2013, Associate Dean for Graduate Education from 2012-2016, and Faculty Director of Online Learning 2012-2017. He obtained his undergraduate degree in Electrical Engineering from the National Technical University of Athens, 1986, and his PhD in Computer Science from the University of Karlsruhe, 1992. He is co-recipient of the Best Conference Paper Award at ICRA 2017 and Best Paper Finalist at IEEE CASE 2015, RSS 2018, and CVPR 2019. Kostas’ main interest today is in geometric deep learning, event-based cameras, and action representations as applied to vision based manipulation and navigation.