Introduction

The banking industry has rapidly embraced AI and machine learning models across various operations, with text recognition emerging as a pivotal technology for automating tasks. From streamlining account opening procedures to verifying KYC details, AI-driven solutions are transforming traditional banking processes.

A critical application of this technology lies in automating the validation of vendor invoices for payment disbursement. Historically, this has been a manual and time-consuming process, where teams meticulously cross-check key invoice metrics—such as tax details, PAN numbers, and GST information—against existing system records. By leveraging AI in this domain, banks are poised to significantly enhance efficiency, reduce errors, and accelerate payment cycles.

Current Challenges in Invoice Validation

The invoice validation process in our banking operations is currently facing significant challenges due to its manual nature and high volume. The following flowchart illustrates the current manual process:

Our current invoice validation process involves the following steps:

1. Receive 4000-5000 invoices daily from various vendors across India.
2. Manually check each invoice against system records, a time-consuming task performed by a team of checkers.
3. Match invoice details with tax masters to verify tax-related information.
4. Based on the manual checks, invoices are either verified or flagged for discrepancies.

This manual approach presents several challenges:

• High volume of invoices creates a substantial workload for the checking team.
• Time-intensive process leads to potential bottlenecks in payment disbursement.
• Manual checking is prone to human error, especially given the repetitive nature of the task.
• Significant human resources are consumed, which could be allocated to higher-value tasks.
• The time-consuming nature of the process can lead to delays in vendor payments.

Solution Development

To address the challenges presented by our current manual invoice validation process, we started on a journey to develop an automated solution. This section outlines our proposed solution and the steps we took to arrive at our final approach.

Proposed Solution

Our proposed solution aims to automate the invoice validation process using AI and OCR technologies. Here's a visual representation of how this solution compares to our current process:

The proposed automated process involves the following steps:

1. Receive 4000-5000 invoices daily (unchanged from the current process).
2. Automatically fetch scanned images of invoices and corresponding system records.
3. Utilize Optical Character Recognition (OCR) technology to extract relevant details from the scanned invoices.
4. Output cross-checked results with match rates, quickly identifying verified invoices and potential discrepancies.

This automated approach is designed to streamline the invoice validation process, minimize manual effort, and improve the accuracy and efficiency of invoice processing.

Initial Challenges

Before we describe the implementation of our final solution, it's important to understand the major challenges we faced at the outset of this project:

1. Varying Invoice Formats:

   • Invoices came from multiple vendors, each with their own format.
   • There was no fixed number of pages or consistent placement of metrics.
   • For example, Vendor A might use a table format with the company name at the top and GST values at the bottom, while Vendor B might use a paragraph format with different placements for key information.

2. Image Quality Issues:

   • While most invoices were printed, many contained handwritten entries, corrections, stamps, or other disturbances that affected image quality.
   • These inconsistencies created significant noise in the data, making extraction more challenging.

3. Complex Metric Extraction:

   • Key metrics included invoice number, date, tax details (SGST, CGST, IGST), IRN, HSN codes, PAN, and GST numbers.
   • While some values like invoice numbers were relatively easy to identify, others like GST appeared multiple times throughout documents.
   • The position of values relative to their labels was inconsistent (above, below, or next to the label).
   • For invoices with multiple products, there were multiple HSN code entries to handle.

4. India-Specific Metrics:

   • Metrics like GST, PAN, and HSN are specific to India.
   • Many pre-built models are not trained on these India-specific metrics, limiting their effectiveness for our use case.

These challenges significantly impacted the performance of standard OCR solutions. For instance, despite efforts in image preprocessing (enhancing image clarity, spell checking, and adjusting text spacing), our initial approach using EasyOCR only achieved an overall accuracy of 39%.

Implementing the Solution

Our path to this proposed solution involved exploring several approaches, each building upon the lessons learned from the previous one.

1. Initial Approach: OCR Libraries

We initially experimented with various OCR libraries:

• Tesseract
• EasyOCR
• OpenCV for image pre-processing

We favored EasyOCR due to its ease of implementation and support for multiple languages. However, this approach faced significant challenges:

• Varying invoice formats from different vendors
• Image quality issues, including handwritten entries and stamps
• Difficulty in extracting complex, India-specific metrics (GST, PAN, HSN codes)

Despite efforts in image preprocessing, this approach only achieved an overall accuracy of 39%.

2. Microsoft Azure Document Intelligence: Basic "Read" Model

Recognizing the limitations of our initial approach, we turned to Microsoft Azure Document Intelligence. We first tested their basic "Read" model to extract text directly. However, this proved insufficient due to:

• Inconsistencies in formatting
• Challenges in metric placement
• Difficulty handling the widespread occurrence of common terms like "GST"

3. Microsoft Azure Document Intelligence: Pre-built Invoice Models

Next, we experimented with Azure's pre-built invoice models. While these improved overall performance, they struggled with:

• Crucial India-specific metrics (GST, PAN, HSN codes)
• Adapting to the unique formats of Indian invoices

4. Custom Model Development

To overcome the limitations of both our initial approach and the pre-built Azure models, we developed a custom model tailored to our specific needs:

1. Data Preparation: We manually annotated over 200 invoices, marking required fields and their locations.
2. Model Training: Using Azure Document Intelligence's custom model capabilities, we trained a new model on our annotated dataset.
3. Iterative Refinement: We went through several rounds of testing and refinement, continuously improving the model's accuracy.

This custom approach proved to be the key to overcoming our unique challenges, significantly improving our ability to accurately process and validate invoices at scale.

In the next section, we'll discuss the results and impact of our custom model implementation.

Results and Impact

The development and implementation of our custom model using Microsoft Azure Document Intelligence led to significant improvements in our invoice validation process. Here, we present the key results and their impact on our operations.

Performance Comparison

To evaluate the effectiveness of our solution, we compared the performance of different approaches on a sample of 25 invoices. The following table summarizes the recall for key fields extracted from the invoices using the Pre-Built Invoice Model:

Next, we have the results from the custom model:

As seen from the results, our custom model demonstrated significant improvements over the pre-built invoice model, particularly in handling India-specific fields. Here are the key findings from our performance comparison:

1. Consistent High Performance: Both models showed excellent recall (100%) for basic fields like PAN No. and Invoice Date.

2. Improved HSN Code Extraction: The custom model dramatically improved HSN code recognition from 19% to 88% recall, a critical enhancement for Indian invoices.

3. New Capabilities: The custom model introduced the ability to extract CGST, SGST, and IGST separately, which wasn't possible with the pre-built model.

4. Slight Decrease in Invoice Number Recall: We observed a minor decrease in recall for the invoice number, from 100% in the pre-built model to 90% in our custom model. While this is a slight reduction, the overall benefits in other areas, particularly in India-specific fields, outweigh this small decrease.

These results demonstrate that our custom model significantly outperforms the pre-built model in handling the complexities of Indian invoices, especially in tax-related information extraction. The slight decrease in invoice number recall is an area for potential future improvement, but it does not detract from the model's overall enhanced performance.

Key Improvements and Operational Impact

Our custom model brought about significant improvements in invoice processing, particularly in handling India-specific metrics like GST, HSN codes, and PAN numbers. It demonstrated remarkable adaptability to diverse invoice formats from various vendors and enabled granular data extraction, including specific tax breakdowns that were previously unattainable with pre-built models.

The operational impact of implementing this custom model has been substantial. Processing time for each invoice has been slashed from 15 minutes to approximately 5 minutes, significantly improving overall efficiency. This reduction, coupled with higher accuracy rates, has drastically decreased the need for manual verification and correction. As a result, our team of 30 checkers can now manage the daily influx of 4,000-5,000 invoices more effectively, freeing up valuable resources for other critical tasks. The streamlined process has accelerated our payment cycle, reducing the turnaround time for payment disbursal from 4-5 days to just 2-3 days. This improvement not only enhances vendor relationships but also potentially qualifies us for early payment discounts. Moreover, the automated system's scalability allows us to handle increased invoice volumes without a proportional rise in processing time or resources, positioning us well for future growth and fluctuations in workload.

Conclusion and Future Work

Our project to automate invoice validation has successfully transformed a manual, time-intensive process into an efficient, scalable system. By developing a custom extraction model using Microsoft Azure Document Intelligence, we've dramatically improved our operations:

• Processing time per invoice reduced from 15 minutes to 5 minutes
• Overall payment disbursal turnaround time decreased from 4-5 days to 2-3 days
• Significantly improved accuracy in handling diverse Indian invoice formats

These improvements have optimized resource allocation, accelerated our payment cycle, and enhanced vendor relationships.

Looking ahead, we are exploring cost-effective machine learning models and hybrid approaches combining OCR and Natural Language Processing (NLP) techniques. These investigations aim to further enhance our system's ability to recognize and process complex invoice formats, ensuring we remain adaptable to future challenges in financial operations.

References

1. Tesseract OCR

   • Smith, R. (2007). An Overview of the Tesseract OCR Engine. Ninth International Conference on Document Analysis and Recognition (ICDAR 2007)
   • GitHub Repository: https://github.com/tesseract-ocr/tesseract

2. EasyOCR

   • JaidedAI. (2020). EasyOCR: Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.
   • GitHub Repository: https://github.com/JaidedAI/EasyOCR

3. OpenCV (Open Source Computer Vision Library)

   • Bradski, G. (2000). The OpenCV Library. Dr. Dobb's Journal of Software Tools
   • Official website: https://opencv.org/

4. Microsoft Azure Document Intelligence

   • Microsoft. (2023). Azure AI Document Intelligence documentation
   • Documentation: https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/?view=doc-intel-4.0.0

5. Optical Character Recognition (OCR) Technology

   • Mori, S., Suen, C. Y., & Yamamoto, K. (1992). Historical review of OCR research and development. Proceedings of the IEEE, 80(7), 1029-1058.