This project presents a cutting-edge AI-powered solution to monitor and deter smoking in restricted areas. By leveraging YOLOv8 for real-time smoker detection, facial recognition for individual identification, and email integration for instant notifications, the system ensures prompt and effective enforcement of no-smoking policies. This innovative approach addresses public health and safety concerns, offering a robust tool for security teams, facility managers, and public health authorities. With seamless integration of state-of-the-art technologies, the system sets a new standard in automated surveillance and regulation enforcement
This project addresses the critical challenge of monitoring and enforcing no-smoking policies in restricted areas. By leveraging advanced AI technologies, the system detects individuals smoking in unauthorized zones and integrates with a facial recognition system to identify the employee involved. The system automatically retrieves the detected zone information and the manager's address, sending detailed email notifications to the relevant departments. Each notification includes comprehensive details such as the time of the incident, the employee's name, the restricted area, and the manager's contact information, along with an attached image as evidence.This innovative approach enhances accountability, reduces manual monitoring efforts, and provides robust support for regulatory compliance in sensitive environments.
This project employs a combination of advanced AI technologies and custom algorithms to detect smokers in restricted areas, identify individuals through facial recognition, and send real-time email alerts to relevant departments.
1.Smoker Detection with YOLOv8
The system utilizes YOLOv8 (You Only Look Once version 8) for real-time object detection to identify smokers in restricted zones. YOLOv8's deep learning model processes video streams or images to detect smoking behavior with high accuracy and speed.
2. Facial Recognition with FaceNet and MTCNN
Upon detecting a smoker, the system uses MTCNN (Multi-task Cascaded Convolutional Networks) to perform facial detection, ensuring that the correct face is isolated from the frame. Next, FaceNet is used to extract and compare facial features to identify the individual. If the person is recognized, their information (such as name) is retrieved from a database.
3. Vector Database with ChromaDB
ChromaDB is utilized as a vector database to store the facial feature vectors generated by FaceNet. This allows the system to efficiently search for and retrieve recognized individuals' information, including their associated employee details and manager contact information.
4.Email Alerts
Once a smoker is identified, the system sends an automated email alert to the required department. The email includes relevant details, such as the time of the incident, the employee’s name, the detected zone, and an image of the smoker as evidence. Python’s smtplib library is used for seamless email integration and communication.
This section provides the implementation details for the core components of the project. The code is divided into three main steps: building a facial recognition system, preparing a vector database, and training YOLOv8 for smoker detection. Each step outlines the corresponding functionality and demonstrates how the components work together seamlessly.as below :-
1.Create Facial Recognition System
Utilize ChromaDB to store facial embeddings as vectors.
Implement functions to add, query, and retrieve data for efficient identification.
3.Train YOLOv8 for Smoker Detection
Train YOLOv8 on a custom dataset to detect smoking behavior in restricted areas.
Integrate YOLOv8 with the system for real-time detection and trigger subsequent processes
MTCNN (Multi-task Cascaded Convolutional Networks)
self.detector = MTCNN()
self.faceNetModel = FaceNet()
in this step we need to extract faces from earch face and generate Embbeding as below :-
class FACELOADING: def __init__(self, directory): self.directory = directory self.target_size = (160, 160) self.X = [] self.Y = [] self.EMBEDDED_X = [] # save face embeddings self.detector = MTCNN() self.faceNetModel = FaceNet() def extract_face(self, filename): img = cv2.imread(filename) img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) x, y, w, h = self.detector.detect_faces(img)[0]['box'] face = img[y:y+h, x:x+w] face_arr = cv2.resize(face, self.target_size) return face_arr.astype('float32') def load_faces(self, folderName): FACES = [] for im_name in os.listdir(folderName): try: path = os.path.join(folderName, im_name) single_face = self.extract_face(path) FACES.append(single_face) except Exception as e: pass return FACES def load_classes(self): for sub_dir in os.listdir(self.directory): path = os.path.join(self.directory, sub_dir) FACES = self.load_faces(path) labels = [sub_dir for _ in range(len(FACES))] print(f"Loaded successfully: {len(labels)} faces for class '{sub_dir}'") self.X.extend(FACES) self.Y.extend(labels) return np.asarray(self.X), np.asarray(self.Y) def get_embedding(self, face_img): face_img = np.expand_dims(face_img, axis=0) yhat = self.faceNetModel.embeddings(face_img) return yhat[0] # 512D image (1x1x512) def run_embedding(self): for img in self.X: self.EMBEDDED_X.append(self.get_embedding(img)) return np.asarray(self.EMBEDDED_X) def aggregate_embeddings(self): embeddings_dict = {} for i, label in enumerate(self.Y): if label not in embeddings_dict: embeddings_dict[label] = [] embeddings_dict[label].append(self.EMBEDDED_X[i]) aggregated_embeddings = {} for label, embeddings in embeddings_dict.items(): print(f'label is '+label) aggregated_embeddings[label] = np.mean(embeddings, axis=0) return aggregated_embeddings if __name__ == '__main__': directory = "D:/openCV/vision_ahmed/projects/enhancedAtendanceSystemFaceNetfullcodewithSpofing/images" face_loader = FACELOADING(directory) X, Y = face_loader.load_classes() face_embedded = face_loader.run_embedding() aggregated_embeddings = face_loader.aggregate_embeddings()
In this project, ChromaDB is utilized as a vector database to efficiently store and manage facial embeddings. Facial embeddings are generated using FaceNet and represent a unique numerical vector for each individual's face. These embeddings are indexed in ChromaDB, enabling fast and accurate retrieval for facial recognition.
import chromadb chroma_client = chromadb.PersistentClient(path="D:/openCV/vision_ahmed/projects//database") #collection = chroma_client.get_or_create_collection(name="my_collection") collection = chroma_client.get_or_create_collection( name="nbeusers", metadata={"hnsw:space": "cosine"} )
To detect cigarettes in real-time, the project leverages a custom-trained YOLOv8 model. The training process involves collecting, annotating, and fine-tuning a dataset specific to cigarette detection. By focusing on this niche category, the model achieves higher accuracy in identifying smoking behavior in restricted zones
email_sender = 'your mail' email_password = 'password' email_receiver = 'your mail'
''' import smtplib import ssl from email.mime.text import MIMEText from email.mime.multipart import MIMEMultipart from email.mime.base import MIMEBase from email import encoders import cv2 import numpy as np from mailConfiguration.email_setting import * from datetime import datetime def sendingMail(from_email=email_sender, to_email=email_receiver, img=None,personName=None): # Get current time and date now = datetime.now() current_time = now.strftime("%H:%M:%S") current_date = now.strftime("%Y-%m-%d") # Create the email headers and subject sent_subject = "Smoking Activity Detected - Immediate Attention Required" # Create the email body sent_body = ( f"This is an automated notification from the SmokerX App. A person identified as {personName} has been detected smoking, " "as captured in the attached image. Below are the details:\n" f"Person Name: {personName}\n" f"Time of Detection: {current_time}\n" f"Date of Detection: {current_date}\n" "Location: Fourth Floor, Building Number 5\n" "Please review the attached image for further verification.\n\n" "Best regards,\n" "SmokerX Monitoring Team" ) # Create a multipart message and set headers message = MIMEMultipart() message["From"] = from_email message["To"] = to_email message["Subject"] = sent_subject # Add body to email message.attach(MIMEText(sent_body, "plain")) if img is not None: # Encode the frame in JPEG format ret, buffer = cv2.imencode('.jpg', img) if ret: image_data = buffer.tobytes() # Create an attachment part image_attachment = MIMEBase("application", "octet-stream") image_attachment.set_payload(image_data) encoders.encode_base64(image_attachment) image_attachment.add_header( "Content-Disposition", "attachment; filename=smoking_detected.jpg", ) # Attach the image to the email message.attach(image_attachment) # Convert the message to a string email_text = message.as_string() try: # Connect to the SMTP server using SSL context = ssl.create_default_context() server = smtplib.SMTP_SSL('smtp.gmail.com', 465, context=context) server.login(email_sender, email_password) # Send the email server.sendmail(email_sender, [email_sender, email_receiver], email_text) # Close the server connection server.quit() print('Email sent successfully!') return True except Exception as exception: print(f"Error: {exception}") return False
def getObjectCoordinates(x_center, y_center, box_w, box_h): x1 = int(x_center - box_w / 2) y1 = int(y_center - box_h / 2) x2 = int(x1 + box_w) y2 = int(y1 + box_h) return x1, y1, x2, y2
def async_send_mail(img, track_id): try: sendingMail(img=img) print(f"Email sent for track ID: {track_id}") except Exception as e: print(f"Failed to send email for track ID {track_id}: {str(e)}")
def async_db_query(emddbedVector, track_id): try: final = collection.query( query_embeddings=emddbedVector.tolist(), n_results=1 ) return final except Exception as e: print(f"Database query failed for track ID {track_id}: {str(e)}") return None
A separate model, fine-tuned for smoking activity or cigarette detection, processes the regions of interest (ROIs) provided by the person detection model.
If smoking behavior is detected within a person’s bounding box, it triggers an alert.
The detections from both models are merged to localize and identify individuals engaged in smoking.
Identified smokers are logged for further actions, such as facial recognition and email alerts.
Outputs include:
Annotated frames highlighting smokers.
Details about detected individuals (name, time, location, etc.).
Evidence in the form of captured images or video clips.
while True: ret, frame = cap.read() if not ret or frame is None: print("Failed to capture image") break #frame = cv2.resize(frame, (640, 480)) frame=cv2.resize(frame,(1020,500)) results = model.track(source=frame, persist=True, tracker="botsort.yaml") if results and results[0].boxes is not None: boxes = results[0].boxes.xywh.cpu() class_ids = results[0].boxes.cls.int().cpu().tolist() track_ids = results[0].boxes.id if track_ids is not None: track_ids = track_ids.int().cpu().tolist() else: print("Warning: No track IDs were returned.") for box, class_id, track_id in zip(boxes, class_ids, track_ids or []): if class_id in target_class_ids: cx, cy, w_, h_ = box x1, y1, x2, y2 = getObjectCoordinates(cx, cy, w_, h_) person_roi = frame[y1:y2, x1:x2] img, bboxs = detector.findFaces(person_roi, draw=False) Smokingresults = smoker.predict(frame, stream=True) if bboxs: for bbox in bboxs: fx, fy, fw, fh = bbox['bbox'] face_x1 = x1 + fx face_y1 = y1 + fy face_x2 = face_x1 + fw face_y2 = face_y1 + fh face = frame[face_y1:face_y2, face_x1:face_x2] if track_id not in personFace: personFace[track_id] = (fx, fy, fw, fh) for r in Smokingresults: boxs = r.boxes for box in boxs: x1_s, y1_s, x2_s, y2_s = box.xyxy[0] x1_s, y1_s, x2_s, y2_s = int(x1_s), int(y1_s), int(x2_s), int(y2_s) w_s, h_s = x2_s - x1_s, y2_s - y1_s conf = math.ceil((box.conf[0] * 100)) / 100 cls = int(box.cls[0]) currentClass = classNames_[cls] if face_x1 <= x1_s <= face_x2 and face_y1 <= y1_s <= face_y2: cv2.rectangle(frame, (x1_s, y1_s), (x2_s, y2_s), (0, 0, 255), 2) cv2.putText(frame, currentClass, (x1_s, y1_s - 20), cv2.FONT_HERSHEY_COMPLEX, 1, (255, 0, 0), 1) cv2.rectangle(frame, (face_x1, face_y1), (face_x2, face_y2), (0, 0, 255), 2) if track_id not in personNames: face_arr = cv2.resize(face, (160, 160)) face_arr = face_arr.astype('float32') face_img = np.expand_dims(face_arr, axis=0) yhat = faceNetModel.embeddings(face_img) yhat = yhat[0] emddbedVector = np.asarray(yhat) # Start a new thread for the database query final = async_db_query(emddbedVector, track_id) if final: score = round(1 - final['distances'][0][0], 2) print(score) if score >= .40: text = final['metadatas'][0][0]['username'] personNames[track_id] = text cv2.putText(frame, str(text), (face_x1, face_y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2) if track_id not in smokerMails: smokerMails[track_id] = 'email sent' # Start a new thread for sending mail threading.Thread(target=async_send_mail, args=(face, track_id)).start() score = f"-{score}%" else: text = 'Unknown' score = '-0%' personNames[track_id] = text cv2.putText(frame, str(text), (face_x1, face_y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2) if track_id not in smokerMails: smokerMails[track_id] = 'email sent' # Start a new thread for sending mail threading.Thread(target=async_send_mail, args=(face, track_id)).start() else: cv2.putText(frame, str(personNames[track_id]), (face_x1, face_y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2) cvzone.overlayPNG(frame, logo, pos=[0, 426]) cv2.imshow('Smoker X', frame) key = cv2.waitKey(10) if key == ord('q'): break cap.release() cv2.destroyAllWindows()
This project successfully demonstrates the integration of advanced AI models to address a real-world problem of smoking in restricted areas. By combining the power of object detection, facial recognition, and email integration, the system provides a comprehensive solution for monitoring and reporting unauthorized smoking behavior.
The dual-model architecture ensures accurate detection of individuals and smoking activities, while the use of ChromaDB for facial embeddings enables swift identification of violators. The email alert system adds a layer of immediacy, ensuring that responsible departments are notified promptly with all necessary details, including time, location, and photographic evidence.
This solution is not only robust and scalable but also adaptable to various environments such as offices, schools, hospitals, and public spaces. The modular design allows for easy updates or extensions, such as adding new detection capabilities or enhancing the user interface.
Moving forward, this project can be expanded to incorporate real-time dashboards, SMS alerts, or integration with IoT devices for automatic response mechanisms. It serves as a model for leveraging AI in safety and compliance monitoring, setting the foundation for more intelligent surveillance systems in the future.