Abstract

Autonomous driving research demands robust solutions for object detection and tracking in dynamic environments. This project integrates YOLOv9, a state-of-the-art object detection algorithm, with DeepSORT for real-time multi-object tracking in the CARLA Simulator. By combining these technologies, the system can accurately identify and follow objects across frames, supporting applications like trajectory planning and decision-making. The results demonstrate the efficiency of this pipeline, achieving high detection accuracy and reliable tracking under various simulated conditions. This publication outlines the methodology, experimental setup, and results, providing a foundation for further research in autonomous systems.

1. Introduction
   Autonomous driving research relies heavily on accurate and efficient object detection and tracking systems. Traditional approaches often struggle with real-time performance, occlusion handling, and adaptability to complex driving scenarios. This study addresses these challenges by:

a. Implementing YOLOv9 for high-precision object detection
b. Utilizing DeepSORT for robust multi-object tracking
c. Leveraging CARLA Simulator for comprehensive performance evaluation

1.1 Research Objectives

a. Develop a seamless integration of YOLOv9 and DeepSORT
b. Evaluate tracking performance in simulated driving environments
c. Demonstrate real-time object detection and tracking capabilities

Methodology

1. System Overview
   The project pipeline consists of the following components:

a. CARLA Simulation Environment: Provides high-fidelity urban and rural settings for testing.
b. YOLOv9 Object Detection: Identifies objects such as vehicles, pedestrians, and traffic signs.
c. DeepSORT Tracker: Tracks detected objects across video frames.
Integration with Python and OpenCV: Facilitates real-time visualization and analysis.

2. Implementation Steps
   Data Collection in CARLA
   -The CARLA Simulator generates synthetic video data. The trajectory_planning.py script simulates autonomous vehicle navigation and records footage.
   Object Detection with YOLOv9
   -YOLOv9 is fine-tuned on datasets relevant to autonomous driving. Detection is performed on video frames extracted from CARLA.

YOLOv9 object detection example

import argparse
import os
import platform
import sys
from pathlib import Path
import math
import torch
import numpy as np
from deep_sort_pytorch.utils.parser import get_config
from deep_sort_pytorch.deep_sort import DeepSort
from collections import deque
FILE = Path(__file__).resolve()
ROOT = FILE.parents[0]  # YOLO root directory
if str(ROOT) not in sys.path:
    sys.path.append(str(ROOT))  # add ROOT to PATH
ROOT = Path(os.path.relpath(ROOT, Path.cwd()))  # relative

from models.common import DetectMultiBackend
from utils.dataloaders import IMG_FORMATS, VID_FORMATS, LoadImages, LoadScreenshots, LoadStreams
from utils.general import (LOGGER, Profile, check_file, check_img_size, check_imshow, check_requirements, colorstr, cv2,
                           increment_path, non_max_suppression, print_args, scale_boxes, strip_optimizer, xyxy2xywh)
from utils.plots import Annotator, colors, save_one_box
from utils.torch_utils import select_device, smart_inference_mode

def initialize_deepsort():
    # Create the Deep SORT configuration object and load settings from the YAML file
    cfg_deep = get_config()
    cfg_deep.merge_from_file("deep_sort_pytorch/configs/deep_sort.yaml")

    # Initialize the DeepSort tracker
    deepsort = DeepSort(cfg_deep.DEEPSORT.REID_CKPT,
                        max_dist=cfg_deep.DEEPSORT.MAX_DIST,
                        # min_confidence  parameter sets the minimum tracking confidence required for an object detection to be considered in the tracking process
                        min_confidence=cfg_deep.DEEPSORT.MIN_CONFIDENCE,
                        #nms_max_overlap specifies the maximum allowed overlap between bounding boxes during non-maximum suppression (NMS)
                        nms_max_overlap=cfg_deep.DEEPSORT.NMS_MAX_OVERLAP,
                        #max_iou_distance parameter defines the maximum intersection-over-union (IoU) distance between object detections
                        max_iou_distance=cfg_deep.DEEPSORT.MAX_IOU_DISTANCE,
                        # Max_age: If an object's tracking ID is lost (i.e., the object is no longer detected), this parameter determines how many frames the tracker should wait before assigning a new id
                        max_age=cfg_deep.DEEPSORT.MAX_AGE, n_init=cfg_deep.DEEPSORT.N_INIT,
                        #nn_budget: It sets the budget for the nearest-neighbor search.
                        nn_budget=cfg_deep.DEEPSORT.NN_BUDGET,
                        use_cuda=True
        )

    return deepsort

Object Tracking with DeepSORT
-DeepSORT associates YOLOv9 detections across frames to maintain object identity.

Results

1. Object Detection Accuracy
   -The YOLOv9 model achieved an average precision (AP) of 85.3% for vehicles and pedestrians in CARLA-generated scenarios.

2. Tracking Consistency
   -DeepSORT successfully tracked 92% of detected objects across 1000 frames, maintaining a consistent track ID.
3. Real-Time Performance
   -The system processed video at an average of 24 FPS on an Intel i7 processor, 16 GB RAM.

Explore the Code
GitHub Repository: https://github.com/ROBERT-ADDO-ASANTE-DARKO/YOLOv9-DeepSORT-realtime-object-tracking-CARLA