In the field of autonomous systems, achieving reliable navigation through complex environments is a challenging task. In this project, we focus on designing and implementing an autonomous navigation and track-following system using computer vision techniques and machine learning algorithms. The system employs a Pixy Camera for real-time image capture and processing, which feeds into a computer vision pipeline built on ROS2 (Robot Operating System 2). The project aims to demonstrate the integration of vision-based control with dynamic steering mechanisms for autonomous cars.
This system’s architecture can be divided into two primary layers: the CV and ML layer for image processing and object detection, and the Line Following layer for control of the vehicle’s movement based on visual feedback.
The project is organized into several critical layers, each focusing on a specific functionality:
CV and ML Layer:
Responsible for finding major obstacles/directions/traffic signs and signals.
Contribute to finding a safe global route
Line Following Layer:
Find the bounds for the immediate road.
Contributes to finding a safe local route
The integration of these components ensures that the self-driving car can autonomously follow a track, navigate efficiently, and perform adaptive steering based on visual inputs.
The CV and ML Layer is primarily responsible for image processing, feature extraction, and object recognition using the Pixy Camera to interpret the environment. This layer is key for enabling the system to "see" and respond to the track.
At the core of the CV layer is the vision module, which is responsible for processing the raw camera feed from the Pixy camera. The pipeline begins by capturing the camera feed in real-time and converting it into a format suitable for processing by the computer vision algorithms.
Pixy Camera Integration: The Pixy camera, a powerful vision sensor designed for robotics applications, is interfaced with ROS2 to provide high-speed object tracking. It is particularly effective in detecting colored objects, such as the lines on the track.
Image Preprocessing: The raw images captured by the Pixy Camera undergo preprocessing steps to enhance the image quality and extract relevant features. This typically includes:
Feature Detection: Once the image is preprocessed, we extract key features such as:
To enhance the robustness of the vision system, machine learning techniques are employed for adaptive learning and decision-making. The ML models are used to predict the optimal steering angles or vehicle maneuvers based on the visual data.
Supervised Learning: The system is trained using labeled datasets that include images of the track with corresponding steering commands. This enables the model to predict the correct steering direction based on the current visual input.
Decision Models: After processing the image, the system employs a decision-making model (possibly a simple feed-forward neural network) that outputs a steering command to the vehicle. This command adjusts the vehicle's direction in real-time.
The Line Following Layer is the second core component of the system, which processes the output from the computer vision layer and converts it into actionable control signals for the vehicle.
The line-following algorithm is the driving force behind the vehicle’s navigation along the track. The system continuously adjusts the steering based on the detected line to keep the vehicle on course.
Steering Control: The primary task of this layer is to interpret the output from the computer vision system and convert it into motor control signals. The steering angle is dynamically adjusted to ensure the vehicle stays aligned with the track.
The following steps are typically involved:
To improve the robustness of the line-following system, predictive models are used to forecast the path ahead based on the current steering angle and position of the car on the track. This helps the car to anticipate turns and curves, adjusting the steering smoothly rather than reacting too late.
The system is built on ROS2 (Robot Operating System 2), which facilitates communication between different layers of the system. ROS2 provides a modular framework that allows for the integration of sensor data, control algorithms, and actuator commands.
This self-driving car project demonstrates the integration of computer vision and machine learning techniques for autonomous navigation. The use of the Pixy Camera for image capture and the implementation of a ROS2 framework ensures a scalable and real-time system capable of tracking and following a line while adapting to changes in the track. The system can be expanded to include more advanced features, such as obstacle avoidance, path planning, and multi-sensor fusion.