# Abstract This study presents the Posture Estimation-based Unsupervised Spatial Gaussian Clustering (PE-USGC) model to classify human motions, specifically near-duplicate movements with subtle differences. PE-USGC focuses on body landmarks and spatiotemporal patterns, achieving higher accuracy than traditional pixel-based convolutional neural networks (CNN). The posture-based approach improved performance, with an accuracy of 76.3% at 1 FPS to 96.97% at 30 FPS. Posture-based task classification outperformed pixel-based classification by 7.2% and required a lower image frame rate to achieve higher accuracy. PE-USGC was benchmarked on the UCF Sports Action dataset. Here is a link to the [PE-USGC paper](https://doi.org/10.1109/ACCESS.2024.3491655), published in IEEE Access. # Introduction The following video presents an introduction to how PE-USGC works: :::youtube[Title]{#1yVW5Dxf3lw} # # # # Methodology The PE-USGC model classifies human motions by analyzing spatiotemporal patterns from body landmarks. It preprocesses motion data to compute kinematic features, clusters them using Gaussian Mixture Models (GMMs), and reduces dimensionality with Principal Component Analysis (PCA). A Random Forest classifier then categorizes motions with high accuracy by leveraging detailed frame-to-frame spatiotemporal information. ### Data Processing Participants performed tasks like chopping, sawing, and slicing. Key body landmarks were extracted, and velocity and acceleration features were computed. ```python def preprocess_data(landmarks, step_size): velocities = (landmarks[1:] - landmarks[:-1]) / (0.033 * step_size) accelerations = (velocities[1:] - velocities[:-1]) / (0.033 * step_size) features = np.hstack([landmarks[:-2], velocities[:-1], accelerations]) return features ``` ### GMM Clustering Feature vectors were clustered using GMMs for probabilistic representation. ```python from sklearn.mixture import GaussianMixture def train_gmm(features, n_components): gmm = GaussianMixture(n_components=n_components, random_state=42) gmm.fit(features) return gmm ``` ### Dimensionality Reduction PCA reduced the dimensionality of feature vectors while retaining important variance. ```python from sklearn.decomposition import PCA def reduce_dimensionality(features, n_components=2): pca = PCA(n_components=n_components) reduced_features = pca.fit_transform(features) return reduced_features ``` ### Classification A Random Forest classifier categorized motions based on the extracted features. ```python from sklearn.ensemble import RandomForestClassifier def train_classifier(X_train, y_train): rf = RandomForestClassifier(n_estimators=100, random_state=42) rf.fit(X_train, y_train) return rf ``` ### Evaluation The model’s performance was assessed using accuracy metrics across multiple runs. ```python from sklearn.metrics import accuracy_score def evaluate_model(classifier, X_test, y_test): y_pred = classifier.predict(X_test) accuracy = accuracy_score(y_test, y_pred) return accuracy ``` The code for PE-USGC is available [here](https://github.com/iyer1729/PE-USGC). # Results The PE-USGC model demonstrated significant improvements in accuracy for classifying near-duplicate human motions, achieving 76.3% at 1 FPS and 96.97% at 30 FPS, outperforming traditional CNN architectures like ResNet-18 and DenseNet in handling subtle motion differences. The results showed that PE-USGC maintained high accuracy with lower frame resolution, improving efficiency for real-world applications, while posture-based task classification outperformed pixel-based classification by 7.2% and achieved higher accuracy with a lower image frame rate. The [PE-USGC paper](http://doi.org/10.1109/ACCESS.2024.3491655) elaborates on detailed comparisons and additional metrics, including precision, recall, and F1 scores, as well as benchmark evaluation. # Citation If you find PE-USGC helpful for your research or projects, please consider citing our paper: ```python @ARTICLE{Iyer2024PEUSGC, author={Iyer, Hari and Jeong, Heejin}, journal={IEEE Access}, title={PE-USGC: Posture Estimation-Based Unsupervised Spatial Gaussian Clustering for Supervised Classification of Near-Duplicate Human Motion}, year={2024}, volume={12}, pages={163093-163108}, doi={10.1109/ACCESS.2024.3491655} } ```