Abstract

This publication presents the development of a Retrieval-Augmented Generation (RAG) pipeline utilizing Langflow, Astra DB, Ollama embeddings, and the Llama3.2 Large Language Model (LLM). The pipeline demonstrates efficient document ingestion, vector storage, and context-aware question answering, exemplified through a custom PDF story, "Shadows of Eldoria."

Introduction

In the realm of artificial intelligence, enhancing the accuracy and relevance of language models is paramount. Retrieval-Augmented Generation (RAG) pipelines address this by integrating retrieval mechanisms with generative models, enabling more informed and contextually appropriate responses. This project showcases the construction of a RAG pipeline using Langflow, a platform that facilitates the design and deployment of such architectures.

Methodology

Key Components

• Document: Shadows of Eldoria, a 3,000-word fictional narrative generated via ChatGPT.
• Vector Database: Astra DB by DataStax, employed for the storage of embeddings.
• Embedding Model: Ollama's all-minilm:latest, producing 384-dimensional vectors.
• LLM: Llama3.2, hosted locally through Ollama.
• Tools: The pipeline is constructed using Langflow.

Workflow Steps

1. Document Ingestion:
   Text is extracted from the PDF and segmented into chunks of 500 characters with an overlap of 100 characters.

2. Embedding Generation:
   These text chunks are converted into 384-dimensional vectors using Ollama embeddings (all-minilm).

3. Vector Storage:
   The generated embeddings are stored in Astra DB within the langflow_test1 database and the pdf collection.

4. Query Processing:
   User inquiries are processed by performing vector similarity searches to retrieve the top five most relevant results.

5. Response Generation:
   A prompt template combines the retrieved context with the user's question, enabling Llama3.2 to generate contextually appropriate answers.

Experiments

To evaluate the pipeline's efficacy, a series of user queries related to "Shadows of Eldoria" were conducted. The system's ability to retrieve pertinent information and generate coherent responses was assessed.

Results

The pipeline effectively processed user queries, retrieving relevant context from the document and generating accurate, context-aware responses. This demonstrates the potential of integrating retrieval mechanisms with generative models to enhance the performance of language models.

Conclusion

The implementation of this RAG pipeline underscores the advantages of combining retrieval systems with generative language models. By leveraging Langflow, Astra DB, and Ollama embeddings, the pipeline achieves efficient document processing and contextually appropriate question answering. This approach holds promise for applications requiring accurate and context-aware AI responses.