Abstract

This work presents a FinOps Conversational Agentic AI System designed to simplify cloud cost management through natural-language interaction. The system combines Retrieval-Augmented Generation (RAG), multi-agent reasoning, and FinOps domain knowledge to deliver insights on cost allocation, optimization, forecasting, and governance. By integrating vector search, LLM reasoning, and automated decision pathways, the system provides an efficient and accessible way for engineering and finance teams to understand and manage cloud expenditure.

Introduction

Cloud spending is increasing across organizations, creating a need for automated systems that provide real-time insights, optimization recommendations, and proactive cost governance. This publication presents a FinOps-focused conversational agentic AI system that enables users to interact with cloud financial data using natural language.
The system integrates Retrieval-Augmented Generation (RAG), multi-agent orchestration, and role-specific workflows to deliver FinOps intelligence. The pipeline includes:

1. Document ingestion and domain knowledge extraction

2. Embedding generation and vector-based retrieval

3. FinOps-specific reasoning agents

4. LLM-driven conversation layer

5. Cost analysis, forecasting, and optimization modules

Key Features

1. RAG pipeline with vector search and semantic reranking
2. Automatic ingestion of .txt documents from the data folder
3. Persistent ChromaDB vector database
4. High-quality HuggingFace MiniLM embeddings
5. Token-based and paragraph-based text chunking
6. LLM fallback chain: OpenAI → Groq → Retrieval-only
7. Clean architecture with separate modules for vector DB and app logic
8. Environment-driven configuration through .env

Installation

git clone https://github.com/Suchi-BITS/ReadyTensor_Project-1.git
cd <project_folder>
python -m venv venv
venv\Scripts\activate # Windows
python -m pip install --upgrade pip
pip install -r requirements.txt

Project Structure

project/
│
├── data/ # Input documents (.txt files)
│ └── *.txt
│
├── chroma_store/ # Auto-persisted ChromaDB files
│
├── src/
│ ├── app.py # Main RAG application
│ └── vectordb.py # Vector DB wrapper (Chroma + HF + reranker)
│
├── .env # Configuration keys and model selection
└── README.md

Run the app

python src/app.py

How it works

1. Loads all .txt documents from the data folder
2. Splits them into optimized chunks
3. Generates embeddings using HuggingFace
4. Stores vectors in ChromaDB
5. Retrieves relevant chunks for a query
6. Reranks them using CrossEncoder
7. Constructs final context
8. Generates an answer using available LLMs

---

Resetting the Vector Store

To clear Chroma:

from vectordb import VectorDB
v = VectorDB("rag_documents", "sentence-transformers/all-MiniLM-L6-v2")
v.clear_collection()

Or delete the chroma_store folder manually.

---

Notes

• Only .txt files are supported for ingestion.
• Restart the application after adding new documents.
• All components run on CPU by default.
• You can switch to larger embedding models if required.

Architecture Overview

1. Document Ingestion
   Loads all .txt files from the data directory.
   Splits documents into meaningful chunks:
   Paragraph-based chunking
   Token-based chunking using RecursiveCharacterTextSplitter

2. Embedding Layer
   Generates normalized embeddings using HuggingFace MiniLM.
   Implemented via langchain-huggingface.

3. Vector Store
   Stores embeddings using Chroma from langchain-chroma.
   Data persists automatically inside the chroma_store directory.

4. Reranking Layer
   Uses a CrossEncoder reranker (ms-marco-MiniLM-L-6-v2) to reorder retrieved documents and improve answer accuracy.

5. LLM Response Layer
   Priority order:
   OpenAI (gpt-4o-mini)
   Groq (llama-3.1-8b-instant)
   Retrieval-only fallback if no API keys are available

6. Console Interface
   Simple REPL-style question-and-answer interface.
   Displays answer, retrieved context, and document sources.

Conclusion

The FinOps Conversational Agentic AI System provides a practical approach to simplifying cloud financial operations. By integrating RAG, vector retrieval, multi-agent reasoning, and LLM-powered conversation, the system makes complex cost analysis accessible and actionable. This work demonstrates that AI-driven FinOps assistants can enhance decision-making, reduce operational effort, and support effective cost governance in cloud-first organizations.