Abstract

Facial recognition systems face significant challenges when identifying individuals wearing masks, a scenario increasingly relevant in the post-pandemic world. This project implements a Siamese Network designed for facial recognition, capable of recognizing faces even when covered by masks. Leveraging a pretrained model, the system generates feature descriptor vectors for each image and stores them in a SQLite database. When presented with a query image, including masked images, the network compares its feature vector to the database entries to identify the closest match.

Introduction

Face recognition has emerged as a pivotal technology in modern computer vision, finding applications in security, healthcare, and personalized services. However, the global adoption of face masks during the Covid-19 pandemic introduced significant challenges for traditional recognition systems. Masks occlude critical facial features, impairing the ability of conventional algorithms to identify individuals accurately.

This project addresses these challenges by implementing a Siamese Network for robust face identification. The network is designed to handle scenarios involving partially covered faces, such as individuals wearing masks. By using a pre-trained backbone, the model efficiently generates feature descriptor vectors for images, enabling precise matching in a database of celebrity images. The SQLite database serves as a flexible and lightweight storage solution, ensuring seamless retrieval and comparison of image descriptors.

While the initial implementation is centered on a celebrity image database, the system's modular design allows for easy adaptation to broader contexts. This includes secure access systems and personalized services, where accurate recognition remains essential despite the presence of occlusions.

Methodology

1. Data Handling

The project uses preprocessed datasets. Images are loaded directly for training and evaluation.

from torchvision.datasets import ImageFolder
from torchvision.transforms import ToTensor
from torch.utils.data import DataLoader

# Example of dataset loading
dataset = ImageFolder(root='path_to_data', transform=ToTensor())
data_loader = DataLoader(dataset, batch_size=32, shuffle=True)

2. Model Architecture

The Siamese Network is implemented with a pretrained ResNet18 backbone.

import torch.nn as nn
import torchvision.models as models

class SiameseNetwork(nn.Module):
    def __init__(self, pretrained_model):
        super(SiameseNetwork, self).__init__()
        # Load the pretrained model
        self.pretrained_model = pretrained_model
        # Replace the last fully connected layer with an identity layer
        self.pretrained_model.fc = nn.Identity()

    def forward_once(self, x):
        # Forward pass through the pretrained model
        output = self.pretrained_model(x)
        return output

    def forward(self, input1, input2):
        # Forward pass for both inputs through the pretrained model
        output1 = self.forward_once(input1)
        output2 = self.forward_once(input2)
        return output1, output2

3. Inference

The inference stage involves generating descriptor vectors from input images which will be stored in a database for subsequent matching.

import torch

# Define the device for computation
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
siamese_net.to(device)

# Set the model to evaluation mode
siamese_net.eval()

# Initialize lists to store descriptor vectors and file paths
descriptor_vectors = []
file_paths = []

# Iterate through batches in the data loader
for batch in data_loader:
    images, paths = batch
    images = images.to(device)
    with torch.no_grad():
        # Forward pass through the Siamese network with image pairs
        outputs = siamese_net(images, images)
    descriptor_vectors.extend(outputs[0].cpu().numpy())
    file_paths.extend(paths)

4. Database Management

SQLite is utilized for storing the feature vectors.

import sqlite3

# Connect to SQLite database
conn = sqlite3.connect('database.db')
cursor = conn.cursor()

# Create a table to store image paths and descriptor vectors
cursor.execute('''CREATE TABLE IF NOT EXISTS ImageDescriptors
                  (id INTEGER PRIMARY KEY,
                   file_path TEXT,
                   descriptor_vector BLOB)''')

Experiments

The performance of the Siamese Network was evaluated by testing its ability to identify a masked image of an individual against a database containing their unmasked image.

Experimental Setup

1. Database Initialization:

• An unmasked image of the author was uploaded to the database. The system processed the image to generate its feature vector, which was stored in the SQLite database.

2. Verification Test:

• A separate image of the author wearing a mask was used for verification,
• The feature vector of the masked image was generated and compared to all vectors in the database.
• The system then evaluated the similarity between the masked image and the database entries to identify the closest match.

Results

• The system correctly identified the masked image as matching the unmasked image of the author stored in the database.
• The confidence of the match was measured at 82%, demonstrating the system's ability to handle the images effectively.

Analysis

This result highlights the effectiveness of the Siamese Network in recognizing individuals. The identification of the masked image with an accuracy of 82% underscores the robustness of the pretrained model. While the system showed strong performance, the slight reduction in confidence indicates room for improvement.

Challenges and Limitations

1. Masked Image Accuracy:

While the system achieved an accuracy of 82% for identifying masked images, this represents a decline compared to ideal conditions.

2. Scalability:

As the size of the database grows, the computational cost for comparing feature vectors also increases.

3. Generalization:

The model was trained on a specific dataset, which may limit its ability to generalize to images.

4. Training Time:

One notable challenge is the increased training time required by Siamese Networks compared to traditional neural network architectures and machine learning algorithms.

Future Work

To further enhance the capabilities and versatility of the network, two areas of improvement and exploration are proposed:

1. Incorporating Preprocessing Techniques:

• Introduce preprocessing methods such as resizing, normalization, and augmentation to enable the system to handle non-preprocessed datasets,
• Implement preprocessing steps for images uploaded by users, such as resizing, cropping, random rotation and normalization.

2. Exploring Alternative Architectures:

• Experiment with other pretrained models to improve feature extraction capabilities and reduce computational costs during training and inference.

Conclusion

This project successfully implements a Siamese Network for facial recognition, demonstrating its suitability in identifying individuals even under challenging conditions, such as wearing masks.

The experiment results, achieving an 82% accuracy in recognizing a masked image against an unmasked reference, highlight the system’s practical applicability in real-world scenarios.

Despite its strengths, the project reveals areas for improvement, including preprocessing techniques to accommodate non-preprocessed datasets.

Further Reading

A Comprehensive Guide to Siamese Neural Networks

A Friendly Introduction to Siamese Networks

The Why and the How of Deep Metric Learning