Abstract

This study presents the Posture Estimation-based Unsupervised Spatial Gaussian Clustering (PE-USGC) model to classify human motions, specifically near-duplicate movements with subtle differences. PE-USGC focuses on body landmarks and spatiotemporal patterns, achieving higher accuracy than traditional pixel-based convolutional neural networks (CNN). The posture-based approach improved performance, with an accuracy of 76.3% at 1 FPS to 96.97% at 30 FPS. Posture-based task classification outperformed pixel-based classification by 7.2% and required a lower image frame rate to achieve higher accuracy. PE-USGC was benchmarked on the UCF Sports Action dataset. Here is a link to the PE-USGC paper, published in IEEE Access.

Introduction

The following video presents an introduction to how PE-USGC works:

Methodology

The PE-USGC model classifies human motions by analyzing spatiotemporal patterns from body landmarks. It preprocesses motion data to compute kinematic features, clusters them using Gaussian Mixture Models (GMMs), and reduces dimensionality with Principal Component Analysis (PCA). A Random Forest classifier then categorizes motions with high accuracy by leveraging detailed frame-to-frame spatiotemporal information.

Data Processing

Participants performed tasks like chopping, sawing, and slicing. Key body landmarks were extracted, and velocity and acceleration features were computed.

def preprocess_data(landmarks, step_size):
    velocities = (landmarks[1:] - landmarks[:-1]) / (0.033 * step_size)
    accelerations = (velocities[1:] - velocities[:-1]) / (0.033 * step_size)
    features = np.hstack([landmarks[:-2], velocities[:-1], accelerations])
    return features

GMM Clustering

Feature vectors were clustered using GMMs for probabilistic representation.

from sklearn.mixture import GaussianMixture
def train_gmm(features, n_components):
    gmm = GaussianMixture(n_components=n_components, random_state=42)
    gmm.fit(features)
    return gmm

Dimensionality Reduction

PCA reduced the dimensionality of feature vectors while retaining important variance.

from sklearn.decomposition import PCA
def reduce_dimensionality(features, n_components=2):
    pca = PCA(n_components=n_components)
    reduced_features = pca.fit_transform(features)
    return reduced_features

Classification

A Random Forest classifier categorized motions based on the extracted features.

from sklearn.ensemble import RandomForestClassifier
def train_classifier(X_train, y_train):
    rf = RandomForestClassifier(n_estimators=100, random_state=42)
    rf.fit(X_train, y_train)
    return rf

Evaluation

The model’s performance was assessed using accuracy metrics across multiple runs.

from sklearn.metrics import accuracy_score
def evaluate_model(classifier, X_test, y_test):
    y_pred = classifier.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    return accuracy

The code for PE-USGC is available here.

Results

The PE-USGC model demonstrated significant improvements in accuracy for classifying near-duplicate human motions, achieving 76.3% at 1 FPS and 96.97% at 30 FPS, outperforming traditional CNN architectures like ResNet-18 and DenseNet in handling subtle motion differences. The results showed that PE-USGC maintained high accuracy with lower frame resolution, improving efficiency for real-world applications, while posture-based task classification outperformed pixel-based classification by 7.2% and achieved higher accuracy with a lower image frame rate. The PE-USGC paper elaborates on detailed comparisons and additional metrics, including precision, recall, and F1 scores, as well as benchmark evaluation.

Citation

If you find PE-USGC helpful for your research or projects, please consider citing our paper:

@ARTICLE{Iyer2024PEUSGC,
  author={Iyer, Hari and Jeong, Heejin},
  journal={IEEE Access}, 
  title={PE-USGC: Posture Estimation-Based Unsupervised Spatial Gaussian Clustering for Supervised Classification of Near-Duplicate Human Motion}, 
  year={2024},
  volume={12},
  pages={163093-163108},
  doi={10.1109/ACCESS.2024.3491655}
}