Figure 1: Heritage's mask icon.

Illustration: Heritage live feed.

Abstract

Heritage is an agentic Retrieval-Augmented Generation (RAG) system designed to deliver precise, context-aware answers about Nigerian peoples, their pre-historic arts, and cultural practices. Built with LangGraph, Chroma, and Tavily, it integrates a vectorized knowledge base with real-time web search capabilities. The system leverages state-of-the-art (SOTA) models with tool-calling abilities, accessed via OpenRouter for cost efficiency, though it is not a free service. It features a modular architecture for domain adaptability and a Streamlit interface for user interaction. Evaluations show an 80% accuracy rate on heritage-related queries, with web search latency (P99 at 52.21s) identified as an area for optimization. Developed for the Agentic AI Developer Certification Program 2025, Heritage promotes cultural appreciation through AI-driven exploration.

Introduction

Nigeria’s cultural heritage, encompassing ancient art forms like Nok, Igbo-Ukwu, Ife, and Benin, as well as diverse ethnic traditions, is a rich yet underexplored domain in digital education. Heritage addresses this by providing an AI-powered query system that enables users to explore Nigerian arts and culture through natural language interactions. Developed by 🅱🅻🅰🆀, this project leverages Retrieval-Augmented Generation (RAG) to combine a curated knowledge base with real-time web data, ensuring historical depth and contemporary relevance. The system uses state-of-the-art (SOTA) models with tool-calling capabilities, accessed cost-effectively via OpenRouter, though it is not free to operate due to API usage costs. It is deployed at Heritage on Streamlit Cloud and open-sourced under the MIT License at Github repo.

The objectives are:

1. Facilitate exploration of Nigerian arts and culture via intuitive queries.
2. Deliver accurate, context-aware responses using hybrid retrieval.
3. Provide a modular framework for future domain adaptations.

This publication details the system’s methodology, experimental setup, results, and future improvements, fulfilling the Week 1 deliverables of the Agentic AI Developer Certification Program.

Methodology

Heritage employs a modular Retrieval-Augmented Generation (RAG) architecture, leveraging LangGraph for stateful conversation management and tool orchestration. The system integrates a Chroma vector store, populated with PDF documents from the DATA_PATH directory, for historical and cultural data retrieval. An ensemble retriever, combining a similarity-based retriever (k=5) and an SVM retriever with weights [0.7, 0.3], ensures robust document retrieval. For queries requiring current information, the Tavily search tool fetches real-time web data with advanced search depth. The system uses OpenRouter to access cost-efficient SOTA models (meta-llama/llama-4-scout for primary responses and openai/gpt-4.1-nano for conversation summarization). The Streamlit interface enhances user engagement with a culturally themed UI, featuring dynamic fun facts, a query history table, and interactive charts for query metrics.

System Architecture

The pipeline comprises:

1. Document Ingestion and Vectorization:

• PDFs in ./data/ are ingested using PyMuPDFLoader and chunked with RecursiveCharacterTextSplitter.
• Text embeddings are generated with sentence-transformers/all-mpnet-base-v2 and stored in a FAISS vector store.
• An EnsembleRetriever combines similarity and SVM retrievers (weights: 0.7, 0.3) for robust retrieval.

from langchain_chroma import Chroma
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_community.retrievers import SVMRetriever
from langchain.retrievers import EnsembleRetriever

embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2")
loader = DirectoryLoader(data_path, glob="**/*.pdf", loader_cls=PyMuPDFLoader)
docs = loader.load()
vector_store = Chroma.from_documents(docs, embeddings)
similarity_retriever = vector_store.as_retriever(search_kwargs={"k": 5})
svm_retriever = SVMRetriever.from_documents(docs, embeddings)
ensemble_retriever = EnsembleRetriever(retrievers=[similarity_retriever, svm_retriever], weights=[0.7, 0.3])

2. Query Processing:

• Queries are processed via a LangGraph workflow, using meta-llama/llama-4-scout (OpenRouter) for generation.
• A system prompt ensures focus on Nigerian heritage, with tools for database search, web search (Tavily), and off-topic handling.
• Conversation history is maintained with MemorySaver, with summaries updated every six messages to resolve ambiguities.

3. Web Search Integration:

• Tavily API provides real-time data for queries requiring current information, complementing the static knowledge base.

4. User Interface:

• A Streamlit app offers four pages: Home, About, Chat, and Statistics.
• The Chat page displays query-response pairs in a culturally themed UI, while Statistics visualizes query metrics with Plotly.

Figure 2: Heritage's Streamlit interface, showcasing the Statistics page.

Tools

• search_database: Retrieves excerpts from the vector store.
• web_search: Uses Tavily for current information.
• off_topic_tool: Handles queries outside Nigerian heritage scope.

Experiments

Dataset

The knowledge base consists of PDFs on Nigerian arts and culture in ./data/, covering Nok, Igbo-Ukwu, Ife, Benin, and Esie art forms, cultural practices, and historical figures. Due to licensing, the dataset is not publicly shared but can be extended with compliant sources.

Evaluation Setup

The system was tested with heritage_eval_dataset.csv, containing five queries:

• "In which state was Nok art discovered?"
• "Who discovered the Igbo-Ukwu art archaeological site?"
• "In which state was Esie art discovered?"
• "What century does Ife art date back to?"
• "Who is considered Nigeria’s most famous writer?"

Outputs were compared against expected answers using an LLM-based evaluator openai/gpt-4.1-nano, with results saved to heritage_eval_results.csv.

Implementation Details

• Environment: Python 3.10+, dependencies in requirements.txt.
• APIs: OpenRouter (LLM), Tavily (web search), HuggingFace (embeddings).
• Configuration: Managed via .env with keys for OPENROUTER_API_KEY, TAVILY_API_KEY, and DATA_PATH.
• Models: Utilizes SOTA models with tool-calling abilities, such as meta-llama/llama-4-scout for generation and openai/gpt-4.1-nano for evaluation, accessed via OpenRouter for cost efficiency. Note that Heritage is not free to operate due to these API costs.

Performance Monitoring

Latency was monitored over the last 7 days, capturing P90 (0.229s) and P99 (52.21s) latencies. High-latency outliers were traced to Tavily web searches, with one run peaking at 14.21s.

Figure 3: Langsmith dashboard, showing P99 latency at 52.21s due to Tavily web searches.

Results

Evaluation Results

• Test 1 (Nok Art Location): Correctly identified Plateau State, despite the expected answer’s incorrect mention of Kaduna State. Score: True.
• Test 2 (Igbo-Ukwu Discoverer): Credited the Anozie brothers with Thurstan Shaw’s role, complementing the expected answer (Shaw). Score: True.
• Test 3 (Esie Art Location): Matched the expected answer (Kwara State), with minor location differences. Score: True.
• Test 4 (Ife Art Dating): Incorrectly stated 1910 and 1832, missing the expected 12th–15th century AD. Score: False.
• Test 5 (Wole Soyinka): Accurately identified Soyinka with additional details, aligning with the expected answer. Score: True.

Overall Accuracy:

4/5 (80%). The failure on Test 4 indicates gaps in the knowledge base or retrieval logic for historical dating.

Table 1: Summary of evaluation results.

Latency Analysis

Monitoring revealed a P90 latency of 0.229s and a P99 latency of 52.21s over the last 7 days. High-latency outliers, such as a 14.21s run, were attributed to Tavily web searches, likely due to network delays or API response times. This suggests that while most queries are processed efficiently, web-dependent queries can significantly impact performance.

Conclusion

Heritage effectively provides an AI-driven platform for exploring Nigerian arts and culture, achieving 80% accuracy in evaluations. Its modular RAG architecture, combining Chroma-based retrieval with Tavily web search, ensures flexibility and relevance. However, challenges remain:

• Historical Accuracy: The incorrect dating of Ife art (Test 4) highlights the need for a more comprehensive knowledge base.
• Latency: Tavily web searches cause significant latency outliers (P99 at 52.21s), necessitating optimization strategies like caching or conditional web search triggers.

Future improvements include:

• Expanding the knowledge base with verified historical sources.
• Implementing caching for web searches to reduce latency.
• Adding multilingual support for broader accessibility.
• Refining the ensemble retriever to improve historical accuracy.

Heritage represents a meaningful step toward preserving Nigeria’s cultural legacy through AI, with potential for adaptation to other domains.

The project is accessible at Heritage on Streamlit and its source code at Github repo.