Abstract

This project explores the integration of computer vision with drone control, enabling the DJI Tello drone to recognize human gestures and respond with predefined movements. Leveraging OpenCV, cvzone’s HandTrackingModule and FaceDetectionModule, the system detects hand gestures and facial regions, interpreting gestures to command drone actions like moving up, down, left, or right, and performing flips. The developed framework demonstrates the potential for intuitive, non-contact human-drone interaction, enhancing the usability of drones in scenarios requiring remote and hands-free control.

Introduction

Unmanned Aerial Vehicles (UAVs), such as drones, are becoming increasingly prominent in various fields, from entertainment to industrial applications. A crucial aspect of their usability lies in intuitive control mechanisms. Traditional remote controllers, though effective, may not always be practical in dynamic or sensitive environments. Gesture-based control offers a hands-free, natural interface, allowing operators to interact with drones without physical devices. This project implements gesture-based control for the DJI Tello drone using computer vision techniques, enhancing its accessibility and operational versatility.

Methodology

3.1 System Setup

The system utilizes the DJI Tello drone, which is connected to a computer for streaming video and receiving commands. Key libraries employed include:

OpenCV: For image processing and visualization.

cvzone HandTrackingModule: For detecting and analyzing hand gestures.

cvzone FaceDetectionModule: For locating facial regions in the video feed.

djitellopy: For establishing communication and sending commands to the DJI Tello drone.

3.2 Gesture Detection

The hand gestures are detected using the HandTrackingModule. It identifies the hand's keypoints and determines the state of each finger (up or down). Specific gestures correspond to predefined commands for the drone:

All fingers up: Stop.

Index finger up: Move up.

Thumb up: Move right.

Little finger up: Move left.

Index and middle fingers up: Move down.

Thumb, index, and pinky fingers up: Perform a flip.

3.3 Face Detection and Gesture Validation

The FaceDetectionModule identifies a bounding box around a detected face. A secondary bounding region is defined around the face to determine where gestures should be recognized. Hand gestures are processed only when the hand’s center falls within this defined region, reducing the risk of false detections.

3.4 Drone Commands

Detected gestures are mapped to specific drone commands using the djitellopy library. The system sends movement instructions such as move_up, move_down, move_left, move_right, and flip_left. Real-time visual feedback on the gesture and bounding box status is provided in the video feed.

Experiments

4.1 Experimental Setup

Hardware: DJI Tello drone.

Software: Python environment with OpenCV, cvzone, and djitellopy libraries.

Testing Environment: Indoor spaces to ensure controlled lighting and minimal background distractions.

4.2 Test Scenarios

Gesture Recognition Accuracy:

The system’s ability to detect and interpret various hand gestures was evaluated.

Different lighting conditions and distances between the drone and user were tested.

Drone Responsiveness:

The latency between gesture detection and drone movement was measured.

Commands such as directional movements and flips were validated.

Safety Mechanisms:

The drone’s behavior when gestures were unclear or when multiple hands were visible was observed to ensure stable operation.

Results

Gesture Detection: The system successfully recognized and classified gestures with over 90% accuracy in optimal lighting conditions.

Drone Control: Commands were executed with minimal latency, averaging 0.5 seconds between gesture detection and drone response.

System Robustness: The bounding box validation reduced false positives by ensuring gestures were detected only in the region around the user’s face.

Challenges: Gesture detection accuracy decreased in low-light environments or when the user’s hand moved too quickly. Future improvements could include enhanced gesture tracking and environmental adaptability.

Conclusion

his project demonstrates the feasibility of controlling a drone using hand gestures and computer vision. The integration of hand and face detection modules with real-time drone control showcases an innovative approach to human-drone interaction. Potential applications include search-and-rescue missions, surveillance, and entertainment. Future work could focus on expanding gesture vocabularies, improving environmental robustness, and exploring multi-user interactions.