A Vision-based Deep Learning Platform for Human Motor Activity Recognition

In Proceedings of the 12th International Conference on Modern Circuits and Systems Technologies (MOCAST, To Appear),

, , ,


A Vision-based Deep Learning Platform for Human Motor Activity Recognition by Mobina Mobaraki, Anushree Bannadabhavi, Matthew J. Yedlin, and Bhushan Gopaluni
Fig. 2: Sample output of the pre-trained HPE model; The Gaussian probability of the joints is shown as a heatmap in the left figure. They are converted into coordinates and shown in light blue in the right picture.

Click to enlarge image.


To track the body movement of patients with movement disorders, sensors such as Kinect cameras are not easily accessible. Recently-developed deep learning models, as a subset of Artificial Intelligence (AI), can analyze patients’ behavior from RGB images of smartphones. The Stacked Hourglass model is a novel pose estimation deep learning model which can accurately determine the location of body joints and a long shortterm memory network (LSTM) can determine the corresponding action by analyzing the kinematic behavior of the body joints. This study develops a deep learning model that uses RGB images from the UT-Kinect dataset as input and determines the action performed with 84.14 % accuracy. Specifically, our contributions are: (i) developed the preprocessing pipeline to use stack hourglass model on the UT-kinect dataset (ii) finetuning of the model to handle 20 joints (iii) Added a human action recognition component to accurately classify the actions performed. Our method can be an efficient replacement for the hardly-accessible Kinect cameras and can be used to analyze various diseases with movement disorders.

Read or Download: PDF

Can't find a paper? Create a GitHub issue to request a preprint.

DAIS Lab Publications

Read More: