Built a production WhatsApp AI agent that handles text, images, and voice — with per-customer memory, a private knowledge base, and real-time web browsing. The interesting part is the cost architecture: Gemini File Search eliminates the vector database entirely, semantic memory in MongoDB prevents token bloat from growing conversation histories, and DeepSeek handles the reasoning at a fraction of GPT-4 pricing. The result is a fully capable support agent where the infrastructure cost stays flat as usage scales — no trade-offs in capability, just smarter design choices.
🎬 Live Demo: Watch the agent in action on YouTube
📂 Source Code: github.com/Ilaye32/WhatsApp-customer-support
Most AI agent tutorials end at the prototype. They show you how to spin up a chatbot, hook it to an LLM, maybe bolt on a vector database, and call it a day. What they don't show you is what happens when that prototype hits production and the bills start coming in.
Vector database subscriptions. Token bloat from full conversation replays. Separate embedding pipelines. Managed memory services. Before long, the infrastructure cost of your AI agent rivals the salary of the developer who built it.
This article documents a different approach — a fully production-ready WhatsApp AI agent built with a deliberate eye on cost architecture. It handles text, images, and voice. It remembers every customer. It searches private company documents. It can browse the web in real time. And it does all of this without a paid vector database, without redundant token consumption, and without trading away capability to get there.
Before diving into the architecture, it is worth establishing what this system actually handles in production:
The tools available to the agent are a knowledge base (private company documents), a fast web scraper (Firecrawl), a deep web scraper (Crawl4AI), and the image analyzer (Gemini Vision). The core reasoning is done by DeepSeek Chat, one of the most capable and cost-efficient frontier models available today.
Now, the architecture decisions that make this economically viable at scale.
The most significant infrastructure cost in most RAG (Retrieval-Augmented Generation) systems is the vector database. Services like Pinecone, Weaviate Cloud, and Qdrant Cloud all charge for hosted vector storage and query operations. For a startup or small business deploying a customer support agent, this is often the first line item that becomes painful.
This project eliminates that cost entirely using Gemini File Search.
Rather than maintaining a separate vector store, embedding pipeline, and retrieval layer, the knowledge base tool sends queries directly to Gemini with a FileSearch configuration pointing at a named file store. Google handles the embedding, indexing, and retrieval internally.
response = gemini_client.models.generate_content( model="gemini-2.5-flash-lite", contents=query, config=types.GenerateContentConfig( tools=[ types.Tool( file_search=types.FileSearch( file_search_store_names=["your-store-name"] ) ) ] ) )
The practical implication: you pay only for the Gemini API call you were already making. There is no separate vector DB bill, no embedding infrastructure to maintain, no additional latency from a round-trip to a third-party service. The retrieval is embedded directly into the generation step.
For a business with a product catalogue, FAQ document, or operations manual, this is a meaningful saving that compounds as query volume grows.
This is the subtler and arguably more impactful design decision.
In a standard LangGraph agent with a conversation checkpointer, the full message history is loaded and replayed into context on every turn. For a customer who has had twenty interactions with your support agent over a month, that means sending twenty turns of conversation history to the LLM every time they send a new message. Token costs scale linearly with conversation length, and for high-volume deployments, this becomes expensive quickly.
This project addresses the problem with a two-layer memory architecture:
Layer 1 — Short-term checkpointing via MongoDBSaver. This stores the conversation state per thread (each customer's phone number is a unique thread ID). It provides reliable state persistence and recovery.
Layer 2 — Long-term semantic memory via MongoDBStore with vector indexing powered by Gemini Embedding 2 (3072 dimensions). Rather than replaying an ever-growing raw history, the agent can semantically retrieve the most relevant past context when it is needed.
store = MongoDBStore( collection=collection, index_config=VectorIndexConfig( dims=3072, embed=GoogleGenerativeAIEmbeddings(model="models/gemini-embedding-2") ) )
The result is that long-term memory is not a liability. A customer who first asked about a product six weeks ago does not force the agent to load six weeks of chat history. Instead, relevant memories surface semantically when the current conversation calls for them. The agent stays contextually aware without paying for context it does not need.
There is no meaningful trade-off here. The agent still knows the customer. It still has access to their history. It simply retrieves that history intelligently rather than blindly appending every message to an ever-growing prompt.
The choice of LLM is itself an economic decision. DeepSeek Chat is accessed through an OpenAI-compatible endpoint, which means the integration is trivial — a base URL swap — but the cost profile is significantly different from GPT-4 class models.
The model is configured with a 300-second timeout and streaming enabled, making it suitable for the latency requirements of a real-time messaging product.
Rather than operating separate services for conversation checkpoints and vector memory, both are stored in MongoDB — a database most teams already have. langgraph-checkpoint-mongodb handles conversation state. langgraph-store-mongodb handles the semantic vector store. One connection string, one infrastructure item, two jobs done.
For teams on MongoDB Atlas, the free tier is sufficient for early-stage deployments. The cost curve is gradual and predictable.
WhatsApp users frequently send voice notes, particularly in markets where voice is the preferred communication mode. Ignoring audio would mean ignoring a large segment of potential users.
The audio pipeline handles this end-to-end:
From the agent's perspective, a voice message and a text message are identical. The complexity is absorbed entirely in the service layer.
Image handling follows a similar pattern. When a customer sends an image — a product they want to identify, a damaged item they are complaining about, a competitor's flyer — the image is downloaded and passed to Gemini Vision before reaching the agent:
if image_bytes is not None: image_description = analyze_image.invoke({ "image_bytes": image_bytes, "mime_type": mime_type or "image/jpeg" }) user_message = ( f"[Image Analysis]\n{image_description}\n\n" f"[User's message about the image]\n{user_message}" )
The agent receives a rich text description of the image alongside the customer's caption. It never needs to handle raw image bytes directly — the vision model preprocesses the visual input into language the reasoning model can work with natively.
One of the non-obvious production requirements of WhatsApp webhook integrations is response time. Meta's Cloud API expects a 200 OK response to the webhook POST within a short window, or it will retry the delivery. If the agent takes several seconds to think and respond, the webhook handler must not block.
This is solved by returning immediately and processing the message in a background task:
@router.post("/webhook") async def whatsapp_webhook(request: Request, background_tasks: BackgroundTasks): data = await request.json() background_tasks.add_task(process_whatsapp_message, data) return {"status": "received"}
The webhook acknowledges receipt instantly. The actual processing — media download, transcription, agent invocation, reply — happens asynchronously. This prevents timeout errors and retries under load.
Production systems fail in ways that are not visible in logs. Logfire is instrumented at the FastAPI layer, providing distributed tracing across requests. When a customer reports that the agent gave a wrong answer or took too long to respond, Logfire makes it possible to trace exactly what happened: which tools were called, how long each took, what the agent's intermediate reasoning steps were.
This is not a luxury for a production support agent. It is the difference between debugging in the dark and having a full audit trail.
| Component | Choice | Why |
|---|---|---|
| LLM | DeepSeek Chat | Strong reasoning, cost-efficient, OpenAI-compatible |
| Vision | Gemini 2.5 Flash Lite | Best-in-class image understanding, already in the stack |
| STT | Groq Whisper Large v3 | Fast transcription, reliable multilingual support |
| Knowledge Base | Gemini File Search | Eliminates vector DB cost, no separate retrieval pipeline |
| Embeddings | Gemini Embedding 2 | 3072-dim, high-quality, same API key as vision |
| Memory | MongoDB (two collections) | Single infrastructure item for both checkpointing and vector store |
| Web Scraping | Firecrawl + Crawl4AI | Firecrawl for speed, Crawl4AI for depth — complementary |
| Agent Framework | LangGraph | Production-grade state machines, native checkpointing support |
| Server | FastAPI + Uvicorn | Async-first, fast, well-documented |
| Monitoring | Logfire | Native FastAPI integration, distributed tracing |
The server entry point (run.py) handles Windows-specific event loop policy differences — a detail that matters for developers building on Windows before deploying to Linux. On Linux/macOS, asyncio.run() is used directly. On Windows, a SelectorEventLoop is configured explicitly to avoid compatibility issues with Uvicorn.
For production deployment, the recommended path is a Linux VM (any cloud provider) with the server running on port 80 via a process manager, behind an HTTPS reverse proxy (Nginx or Caddy) for the webhook URL that Meta requires.
The prevailing assumption in AI agent development is that capability requires cost — that better memory means a more expensive vector database, that richer context means more tokens, that production-grade observability means another SaaS subscription.
This project challenges that assumption on every front. Gemini File Search absorbs retrieval into the generation call. Semantic memory surfaces relevant context without inflating prompts. MongoDB unifies two infrastructure concerns into one. DeepSeek delivers frontier reasoning at a competitive price point.
The result is an agent that can genuinely serve a small business at scale — not as a prototype that works until the AWS bill arrives, but as a sustainable system where the cost curve stays flat as usage grows.
The lesson is not specific to this stack. It is a design philosophy: every infrastructure decision in an AI system has a cost implication, and the best architecture is the one that eliminates redundancy without eliminating capability.
The full source code is available at github.com/Ilaye32/WhatsApp-customer-support.
The README covers installation, environment configuration, WhatsApp Cloud API setup, and local CLI testing for development without a live WhatsApp number.
Built with LangGraph, DeepSeek, Gemini, Groq, MongoDB, Firecrawl, Crawl4AI, and FastAPI.