Abstract:
Human behavior analysis applications in the fields of ambient assisted living (AAL) and human security monitoring require continuous video analysis of individuals. Although intelligent systems deployed in these areas are intended to have a positive impact on the persons involved, subsequent continuous monitoring naturally raises ethical concerns and questions about privacy implications. To address these issues, we present a foundation for identity-preserving 3D human behavior analysis. Our main contributions are a fast 3D detection system and a public multimodal dataset. The introduced detection system uses an innovative target assignment scheme to significantly improve performance, especially in challenging scenes with a large number of person-person and person-object occlusions. On our dataset, the system shows superior performance compared to the state-of-the-art in 3D object detection while being lightweight enough for configuration and deployment to edge devices. The dataset is large, at a total of ~85k annotated frames, and is based solely on anonymizing sensor technologies with spatio-temporally aligned depth and thermal sequences. Annotation is provided as 3D bounding boxes, along with pose labels and consistent person IDs for use in tracking. The dataset is designed to be flexible. Data representation in either image view or point clouds and the option for projected 2D bounding boxes, allows use in a variety of 2D or 3D tasks. Target applications of our work are privacy-sensitive domains that require continuous supervision, including ambient assisted living tasks (e.g., motion rehabilitation, fall detection, vital sign detection) and human security monitoring applications, such as construction safety, critical care and correctional facility monitoring.
