A Novel Local-Global Feature Fusion Framework for Body-Weight Exercise Recognition with Pressure Mapping Sensors

Abstract

We present a novel local-global feature fusion framework for body-weight exercise recognition with floor-based dynamic pressure maps. One step further from the existing studies using deep neural networks mainly focusing on global feature extraction, the proposed framework aims to combine local and global features using image processing techniques and the YOLO object detection to localize pressure profiles from different body parts and consider physical constraints. The proposed local feature extraction method generates two sets of high-level local features consisting of cropped pressure mapping and numerical features such as angular orientation, location on the mat, and pressure area. In addition, we adopt a knowledge distillation for regularization to preserve the knowledge of the global feature extraction and improve the performance of the exercise recognition. Our experimental results demonstrate a notable 11 percent improvement in F1 score compared to the baseline 3DCNN for exercise recognition while preserving label-specific features.

Dr. Sungho Suh

Senior Researcher

Human Activity Recognition, Safe and Trusted Human Centric Artificial Intelligence in Future Manufacturing Lines

Prof. Dr. rer. nat. Paul (Pawel) Lukowicz

Professor (W3) “Embedded Intelligence”

Paul Lukowicz is Full Professor of AI at the Technical University of Kaiserslautern in Germany where he is heading the Embedded Intelligence group at DFKI. From 2006 till 2011 he has been full Professor (W3) of Computer Science at the University of Passau. He has also been a senior researcher (“Oberassistent”) at the Electronics Laboratory at the Department of Information Technology and Electrical Engineering of ETH Zurich Paul Lukowicz has MSc. (Dipl. Inf.) and a Ph.D. (Dr. rer nat.) in Computer Science a MSc. in Physics (Dipl. Phys.). His research focus are context aware ubiquitous and wearable systems including sensing, pattern recognition, system architectures, models of large scale self-organized systems, and applications. Paul Lukowicz coordinates the FP7-FET SOCIONICAL projects, is Associate Editor in Chief of IEEE Pervasive Computing Magazine, and has been serving as TPC Chair of a number of international events in the area