Abstract

Filling out government disability payment forms is often a complex and time-consuming process, particularly for individuals with disabilities who may face cognitive, physical, or technical barriers. This paper presents an AI-powered chatbot that assists users in completing these forms using Retrieval-Augmented Generation (RAG). The chatbot retrieves relevant information from user-uploaded PDFs and DOCX files, providing real-time, context-aware assistance. Key features include document parsing, optical character recognition (OCR), conversation memory, and dialogue logging for analysis and improvement. Experimental results demonstrate the chatbot’s effectiveness in improving user experience, accuracy, and efficiency in completing disability forms.

Introduction

Individuals applying for disability benefits often encounter difficulties in navigating complex forms, interpreting legal jargon, and ensuring accuracy in their responses. Traditional assistance methods, such as manual guides or human support, may not always be accessible or efficient. Artificial Intelligence (AI)-powered solutions, particularly chatbots leveraging natural language processing (NLP), offer a scalable alternative to assist users in filling out forms accurately and efficiently. This study introduces a chatbot that utilizes Retrieval-Augmented Generation (RAG) to fetch and generate relevant responses, improving accessibility and ease of form completion for individuals with disabilities.

Methodology

1. Setup and Configuration
   • The system is implemented in Python, using OpenAI's language model for response generation.
   • Environment variables manage API keys securely.
2. Document Processing
   • The chatbot supports PDFs and DOCX files, extracting text and images.
   • Optical Character Recognition (OCR) with Pytesseract enhances text extraction from scanned documents.
   • Text is split into meaningful chunks and stored as vector embeddings using FAISS for efficient retrieval.
3. Chatbot Interaction
   • OpenAI's LLM generates responses based on retrieved context from stored embeddings.
   • Prompt engineering ensures accurate and context-aware guidance.
   • The chatbot maintains conversation history to provide continuity and personalized responses.
4. User Interaction
   • Users engage with the chatbot through a command-line interface.
   • New documents can be uploaded for additional reference.
   • Interactions are logged for performance evaluation and research purposes.
5. Evaluation Metrics
   • Response time is logged for performance analysis.
   • Accuracy is assessed by comparing chatbot responses to correct answers.
   • Users provide feedback ratings to evaluate overall experience.

Experiments

To evaluate the chatbot's performance, a user study was conducted with 30 participants who were asked to complete disability forms with and without the chatbot’s assistance. The experiment measured response time, accuracy, and user satisfaction. Participants provided feedback on clarity, ease of use, and overall effectiveness.

Results

• Response Time: The chatbot-assisted group completed forms 40% faster than the control group.
• Accuracy: Users assisted by the chatbot demonstrated a 25% improvement in correctly filling out form sections.
• User Feedback: 85% of participants found the chatbot useful, citing its ability to clarify complex instructions and retrieve relevant form sections efficiently.

Conclusion

The AI-powered Disability Form Assistant chatbot significantly enhances accessibility and efficiency for users filling out government disability forms. By leveraging Retrieval-Augmented Generation, the chatbot provides accurate and contextually relevant guidance, reducing errors and improving user experience. Future work will explore integrating multilingual support and expanding document compatibility for broader applicability.

References

A. Pesaru, T. S. Gill, and A. R. Tangella, "AI Assistant for Document Management Using Lang Chain and Pinecone," International Research Journal of Modernization in Engineering, Technology and Science, vol. 05, no. 06, pp. 3980, Jun. 2023. doi: https://www.doi.org/10.56726/IRJMETS42630. Available: www.irjmets.com.
I. Radeva, I. Popchev, L. Doukovska, and M. Dimitrova, "Web Application for Retrieval-Augmented Generation: Implementation and Testing," Electronics, vol. 13, no. 7, p. 1361, Apr. 2024. [Online]. Available: https://doi.org/10.3390/electronics13071361.

J. Rausch, O. Martinez, F. Bissig, C. Zhang, and S. Feuerriegel, "DocParser: Hierarchical Document Structure Parsing from Renderings," Department of Computer Science, ETH Zurich, and Department of Management, Technology, and Economics, ETH Zurich. [Online]. Available: johannes.rausch@inf.ethz.ch, octaviom@student.ethz.ch, fbissig@student.ethz.ch, ce.zhang@inf.ethz.ch, sfeuerriegel@ethz.ch.

J. Li, J. Chen, R. Ren, X. Cheng, W. X. Zhao, J.-Y. Nie, and J.-R. Wen, "The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models," arXiv preprint arXiv:2312.10997, 2023. [Online]. Available: arxiv.org

Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, and H. Wang, "Retrieval-augmented generation for large language models: A survey," arXiv preprint arXiv:2312.10997, 2023. [Online]. Available: arxiv.org

K. Muludi, K. M. Fitria, J. Triloka, and S. Sutedi, "Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model," International Journal of Advanced Computer Science & Applications, vol. 15, no. 3, p. 776, 2024.

L. Zhang, L. Chen, S. Xu, L. Bai, J. Niu, and W. Wu, "Design of a Structured Parsing Model for Corporate Bidding Documents Based on Bi-LSTM and Conditional Random Field (CRF)," International Journal of Information Technologies and Systems Approach, vol. 16, no. 2, 2024, doi: 10.4018/ijitsa.320645.

N. Mhatre, K. Motani, M. Shah, and S. Mali, "Donna Interactive Chat-bot acting as a Personal Assistant," International Journal of Computer Applications, vol. 140, no. 10, April 2016.

O. Iwashokun and A. Ade-Ibijola, "Parsing of Research Documents into XML Using Formal Grammars," Research Group on Data, Artificial Intelligence, and Innovations for Digital Transformation, University of Johannesburg, Johannesburg, South Africa. Received: Jul. 13, 2023; Revised: Nov. 30, 2023; Accepted: Jan. 6, 2024; Published: Jan. 31, 2024. Academic Editor: J. Chaki. [Online]. Available: opsygirl@gmail.com.

S. Lafia, D. A. Bleckley, and J. T. Alexander, "Digitizing and parsing semi-structured historical administrative documents from the G.I. Bill mortgage guarantee program," ICPSR, University of Michigan, Ann Arbor, MI, USA.

T. Ansari, P. Arshad, V. Khetan, B. Bembre, and P. Halle, "Conversational AI Assistant," International Journal for Research in Applied Science & Engineering Technology (IJRASET), vol. 10, no. XI, pp. 1169, Nov. 2022. Available: www.ijraset.com.

W. T. Sewunetie and L. Kovács, "Exploring Sentence Parsing: OpenAI API-Based and Hybrid Parser-Based Approaches," IEEE Access, vol. 12, pp. 3360480, Jan. 2024. doi: 10.1109/ACCESS.2024.3360480. Received: Nov. 20, 2023; Accepted: Jan. 11, 2024; Published: Jan. 31, 2024; Current version: Mar. 18, 2024. [Online]. Available: waleligntewabe@gmail.com.
Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang, "Retrieval-Augmented Generation for Large Language Models: A Survey," arXiv preprint arXiv:2312.10997v5, Mar. 2024. [Online]. Available: https://github.com/Tongji-KGLLM/RAG-Survey.