Global Position Prediction for Interactive Motion Capture

Paul Schreiner, Researcher, Rokoko, University of Copenhagen He will demonstrate global position estimation from local pose information, including - A method for reconstructing the global position in motion capture using neural networks where position sensing is poor or unavailable such as in IMU-based motion capture, and the - Performance of the proposed method and its benefits over using heuristics based methods. Inertial Measurement Unit (IMU) Paul presented at the recent 20th annual Symposium on Computer Animation (SCA) 2021. https://computeranimation.org/ https://paulschreiner.netlify.app/publication/globalpositions_sca2021/GlobalPositions_SCA2021.pdf We present a method for reconstructing the global position of motion capture where position sensing is poor or unavailable. Capture systems, such as IMU suits, can provide excellent pose and orientation data of a capture subject, but otherwise need post processing to estimate global position. We propose a solution that trains a neural network to predict, in real-time, the height and body displacement given a short window of pose and orientation data. Our training dataset contains pre-recorded data with global positions from many different capture subjects, performing a wide variety of activities in order to broadly train a network to estimate on like and unseen activities. We compare training on two network architectures, a universal network (u-net) and a traditional convolutional neural network (CNN) - observing better error properties for the u-net in our results. We also evaluate our method for different classes of motion. We observe high quality results for motion examples with good representation in specialized datasets, while general performance appears better in a more broadly sampled dataset when input motions are far from training examples. https://www.rokoko.com/about Silicon Valley ACM SIGGRAPH Meetup https://www.meetup.com/SV-SIGGRAPH/events/281918015/ #MotionCapture #IMU #NeuralNetwork 0:00 Title 0:05 Motivation 1:42 IMU motion capture 2:23 The naïve approach 3:34 Heuristic approaches 4:25 Heuristic approaches - examples 4:54 Hypothesis 5:46 Optical data 7:18 Data flow 7:39 Data flow - windowing 8:11 Data flow - inputs 9:09 Data flow - targets 10:35 U-net architecture 11:47 U-net vs. standard CNN 13:49 Running character 14:46 Zig-zag walk 15:11 Dancing 15:35 Zombie walk 15:48 Failure case 16:47 Conclusions