PixEagle 2.0 is an all-in-one image processing, following, and tracking solution meticulously crafted for the PX4 ecosystem, with potential scalability to ArduPilot. Leveraging robust technologies such as MAVSDK Python, OpenCV, and optional YOLO, PixEagle delivers precise object tracking and autonomous navigation capabilities. The system's modular and extensible architecture empowers users to integrate custom tracking, detection, and segmentation algorithms seamlessly. A modern web-based React Ground Control Station (GCS), powered by FastAPI, facilitates real-time monitoring and control, enhancing user experience. PixEagle 2.0 boasts advanced tracker robustness, multiple following modes, and seamless integration with companion computers like the Raspberry Pi 5, positioning it as a transformative tool in aerial robotics applications spanning surveillance, agriculture, and beyond. Comprehensive documentation is available on the PixEagle GitHub Repository, and demonstration videos can be accessed through the PixEagle YouTube Playlist.
Background
Drone tracking and following systems are pivotal in a multitude of modern applications, including surveillance, agriculture, autonomous deliveries, and industrial inspections. These systems enable drones to autonomously navigate and monitor specific targets or areas, significantly enhancing operational efficiency and reducing the need for constant human oversight.
Problem Statement
Existing drone tracking solutions often encounter challenges related to reliability, adaptability, and user-friendliness. Many systems lack the necessary modularity and extensibility, making it difficult to integrate custom algorithms or adapt to varying mission requirements. Additionally, comprehensive real-time monitoring and control interfaces are frequently absent, hindering operational efficiency and user interaction.
Solution Overview
PixEagle 2.0 addresses these challenges by offering a highly modular and extensible drone tracking and following solution. Designed to seamlessly integrate with the PX4 ecosystem and potentially expand to ArduPilot, PixEagle 2.0 supports real-time image processing and autonomous navigation. The inclusion of a web-based React GCS, powered by FastAPI, provides users with an intuitive interface for real-time monitoring and control. Advanced features such as enhanced tracker robustness and multiple following modes ensure reliable and versatile performance across diverse applications.
Drone tracking and following systems have evolved significantly over recent years, with numerous solutions leveraging computer vision and AI to enhance autonomous capabilities. Notable among these are systems utilizing traditional tracking algorithms like CSRT and modern deep learning-based approaches such as YOLO for object detection.
Traditional Tracking Algorithms
Algorithms like CSRT (Discriminative Correlation Filter with Channel and Spatial Reliability) have been widely adopted for their balance between accuracy and computational efficiency. However, they often face limitations in dynamic environments where objects undergo rapid changes in appearance or occlusion.
Deep Learning-Based Approaches
Deep learning models, particularly YOLO (You Only Look Once), have revolutionized object detection with their ability to accurately identify and localize objects in real-time. When integrated with tracking methodologies like Kalman Filters and SORT (Simple Online and Realtime Tracking), these models enhance the robustness and reliability of drone tracking systems.
Modular and Extensible Systems
Recent advancements emphasize the importance of modularity and extensibility, allowing users to customize and extend system functionalities without overhauling core components. Systems that support the integration of custom algorithms and provide flexible interfaces for real-time monitoring and control are gaining traction for their adaptability to various mission requirements.
PixEagle 2.0 builds upon these foundations by combining traditional and deep learning-based tracking algorithms within a modular architecture, offering enhanced flexibility and robustness compared to existing solutions.
System Architecture
PixEagle 2.0's architecture integrates the Pixhawk flight controller with companion computers such as the Raspberry Pi 5, utilizing MAVLink for seamless communication. The system comprises several key components:
MAVSDK Python: Facilitates MAVLink communication between the drone and PixEagle.
OpenCV: Handles real-time image processing tasks.
YOLO (You Only Look Once): Provides smart auto-detection capabilities for object tracking.
React & FastAPI: Power the web-based Ground Control Station (GCS) for real-time monitoring and control.
MAVLink2REST: Bridges MAVLink communication with RESTful APIs for enhanced integration.
Hardware Components
Pixhawk Flight Controller: Manages flight operations and communication protocols.
Raspberry Pi 5: Serves as the companion computer, handling image processing, streaming, and system control.
CSI Cameras: Capture high-resolution video feeds for real-time processing and tracking.
Software Stack
MAVSDK Python: Facilitates seamless MAVLink communication.
OpenCV: Powers image processing and computer vision tasks.
YOLO: Enables real-time object detection and tracking.
FastAPI: Handles backend server operations for the GCS.
React: Provides a responsive and intuitive frontend for the GCS interface.
Tracking Algorithms
PixEagle 2.0 employs a combination of advanced algorithms to ensure accurate and reliable tracking:
Modified CSRT: Enhances object detection accuracy by integrating particle filters for improved robustness.
Particle Filters: Improve tracking reliability in dynamic and noisy environments.
Template Matching: Facilitates precise object redetection when initial tracking fails.
Kalman Filters: Enable accurate object estimation and prediction for smooth tracking.
Modularity and Extensibility
Designed with a fully modular architecture, PixEagle 2.0 allows users to integrate new tracking, detection, and segmentation algorithms without modifying existing system components. This flexibility encourages continuous improvement and adaptation to evolving mission requirements.
Setup
Initial experiments were conducted using a Raspberry Pi 5 as the companion computer, integrated with a Pixhawk flight controller and CSI cameras. The system was configured to handle real-time image processing, streaming, and control operations, interfacing with the PX4 ecosystem via MAVLink.
Tracking Modes
PixEagle 2.0 was tested across various tracking modes to evaluate its versatility and robustness:
Top-Down Follow Mode: Maintains a consistent altitude and position relative to the target from a top-down perspective.
Forward-Looking Follow Mode: Adapts to different mission requirements by adjusting the drone's trajectory based on the target's movement.
Chase Mode: Enables dynamic pursuit of fast-moving targets, simulating scenarios such as "dogfight" engagements.
Performance Metrics
Key performance metrics assessed during the experiments include:
Tracking Accuracy: Measured the precision of object localization and consistency in following the target.
Latency: Evaluated the system's responsiveness in processing and reacting to target movements.
Reliability: Assessed the system's ability to maintain tracking under varying environmental conditions and target dynamics.
Testing Environment
Experiments were conducted in controlled indoor environments with varying levels of lighting and background complexity to simulate real-world conditions. The system's ability to handle occlusions, rapid target movements, and changes in target appearance was rigorously tested.
Tracking Accuracy
PixEagle 2.0 demonstrated high tracking accuracy across all tested modes. The integration of modified CSRT with particle filters significantly reduced tracking errors, maintaining precise localization of targets even during rapid movements and occlusions.
Latency
The system exhibited low latency in processing and responding to target movements, ensuring smooth and responsive drone behavior. Real-time streaming and control operations were maintained consistently, with negligible delays observed during communication between the Raspberry Pi 5 and Pixhawk flight controller.
Reliability
PixEagle 2.0 maintained reliable tracking under diverse environmental conditions. The implementation of redetection and tracker failure compensation mechanisms ensured continuous operation, even when initial tracking attempts were disrupted. The system's ability to adaptively learn and update features during missions further enhanced its robustness.
User Interface Performance
The web-based React GCS provided an intuitive and responsive interface, enabling real-time monitoring and control with minimal latency. Features such as drag-and-select target tracking and real-time data visualization contributed to an enhanced user experience.
Comparative Analysis
Compared to existing solutions, PixEagle 2.0's modular architecture and combination of traditional and deep learning-based tracking algorithms offered superior adaptability and reliability. The ability to integrate custom algorithms without altering core components positioned PixEagle 2.0 as a more flexible and robust solution in the drone tracking landscape.
Strengths
PixEagle 2.0's primary strengths lie in its modular and extensible architecture, allowing seamless integration of custom tracking and detection algorithms. The combination of modified CSRT, particle filters, YOLO, and Kalman filters ensures high tracking accuracy and robustness in dynamic environments. The real-time streaming and responsive GCS interface further enhance operational efficiency and user experience.
Challenges
During initial testing, challenges were encountered in optimizing the system for real-time performance on the Raspberry Pi 5. Ensuring low latency while handling multiple processing tasks required careful optimization of the software stack and efficient utilization of hardware resources. Additionally, fine-tuning the tracking algorithms to handle diverse target behaviors and environmental conditions was a complex task.
Future Improvements
Future enhancements will focus on further improving tracking and detection robustness, increasing system autonomy, and conducting extensive real-world testing to validate and refine performance. Integrating multi-sensor data fusion will enhance tracking accuracy and reliability, while expanding compatibility with additional drone platforms will broaden PixEagle 2.0's applicability.
Community and Collaboration
PixEagle 2.0's open-source nature fosters community collaboration and continuous improvement. Encouraging contributions and feedback from users and developers will drive innovation and adaptability, ensuring PixEagle remains at the forefront of drone tracking and following solutions.
PixEagle 2.0 emerges as a comprehensive and advanced drone tracking and following solution, combining a modular design with robust tracking algorithms and a user-friendly interface. Its seamless integration with the PX4 ecosystem and companion computers like the Raspberry Pi 5, coupled with versatile tracking modes and real-time monitoring capabilities, positions PixEagle 2.0 as a transformative tool in aerial robotics. By addressing key challenges in reliability, adaptability, and user experience, PixEagle 2.0 sets a new standard for drone tracking systems, with significant applications across surveillance, agriculture, autonomous deliveries, and industrial inspections. Future developments will continue to enhance its robustness and scalability, ensuring PixEagle 2.0 remains adaptable to evolving mission requirements and technological advancements.
I would like to extend my gratitude to the open-source community for their invaluable contributions, which have significantly influenced the development of PixEagle 2.0. Special thanks to the contributors on GitHub for their support and feedback. Additionally, appreciation is due to the team at Ready Tensor for providing the platform to showcase innovative projects like PixEagle.
Prerequisites:
Installation Steps:
git clone https://github.com/alireza787b/PixEagle.git cd PixEagle
bash init_pixeagle.sh
nano configs/config.yaml nano dashboard/.env
bash run_pixeagle.sh
http://127.0.0.1:3001. For remote access, replace 127.0.0.1 with your machine's IP address.While in the video window, use the following keys for control:
t: Select targetc: Cancel selectiony: YOLO detectionf: Start following (offboard mode)d: Try to re-detect targetq: QuitEnsure MAVLink communication is properly set up by following the MAVLink Router installation guide. Run MAVLink2REST:
bash ~/PixEagle/src/tools/mavlink2rest/run_mavlink2rest.sh