Abstract

Object tracking systems are widely used nowadays in different spheres, including:

• Autonomous driving systems
• Sports
• Surveillance systems
  Object tracking is a method of tracking detected objects in subsequent frames using spatial and temporal features. Trackers can be classified based on many categories. Single object tracker (SOT) or multiple object tracker (MOT) can respectively track single or multiple objects simultaneously. Tracking algorithms can perform data association across frames either using object detection or without it.
  Simple Online Realtime Tracking (SORT) is a popular algorithm that allows to make associations between the existing trackings from the previous frames and object detections from the current frame by setting tracking ids to the matching detections.
  The assignment cost matrix is composed of the intersection-over-union (IOU) distance between each detection and all the existing trackings. The assignment is solved optimally using the Hungarian algorithm.

Introduction

Current project performs multiple object tracking for the detected athletes using YOLOv8 models. Optionally it allows to select segmentation/pose estimation tasks for the inference.
It takes the *.mp4 video as an input, preprocesses it depending on the selected model parameters, runs the inference and outputs the resulting video that contains the detection, segmentation, pose estimation and tracking data.

Methodology

The code was written using C++ as the inference is supposed to be executed on a hardware so the processing speed is of a priority.
YOLOv8 was used for the tests as it provides a great performance for the real-time object detection, instance segmentation and pose estimation tasks.
Onnxruntime engine was chosen for model inference as a huge number of different model formats can be converted into ONNX. This allowed to test different YOLO models of different size.
SORT algorithm was implemented to perform detections association between the video frames and track the objects.

Experiments

After a set of experiments the YOLOv8m model was chosen a primary one that provides an optimal detection speed-accuracy tradeoff.
Using CPU provider Onnxruntime engine showed the average inference speed of ~42ms. When CUDA provider had been used the inference speed increased to ~14ms which corresponds to the official Ultralytics benchmark. Same true for the Mean Average Precision (mAP) metrics that was about ~51-53. The tests were run on NVIDIA GeForce RTX 4090 driver.
SOTA algorithm showed its weakness in case of the object occlusion. Object ids switch often happened in this scenario and demonstrated a necessity of Deep SORT algorithm implementation to rectify the problem.

Results

Videos for the above examples were taken at pexels.com

Conclusion

YOLO models are considered to be a SOTA for the video processing and they proved to be a such after the set of experiments. It's a great choice for the real-time object detection/instance segmentation/pose estimation tasks.