Abstract

This repository presents a production-grade multi-agent system (MAS) built on the PraisonAI Agents framework, implementing a hierarchical orchestration pattern for automated software development workflows. The system employs five specialized AI agents that collaborate through a structured decision-making process to transform high-level user requirements into fully functional, tested, and deployed software solutions. The architecture incorporates enterprise-grade features including caching, load balancing, monitoring, security, and optimization mechanisms, making it suitable for both research and production deployments.

---

Overview

- System Purpose

The Multi-Agent System (MAS) is designed to automate the complete software development lifecycle through intelligent agent collaboration. The system accepts natural language requirements and orchestrates multiple specialized agents to produce production-ready code, comprehensive test suites, and deployment configurations.

- Key Capabilities

• Requirement Elicitation: Interactive conversation interface for gathering and refining user requirements
• Automated Planning: Generation of detailed implementation specifications in structured JSON format
• Code Generation: Production of executable Python code with proper structure and best practices
• Automated Testing: Comprehensive test suite generation and validation
• Deployment Automation: Containerized deployment configurations (Docker, Kubernetes)
• Enterprise Features: Caching, monitoring, load balancing, security, and cost optimization

- Target Audience

This system is designed for:

• AI Researchers: Studying multi-agent systems, hierarchical orchestration, and agent collaboration patterns
• Software Engineers: Implementing automated development workflows and CI/CD pipelines
• DevOps Practitioners: Deploying and managing AI-powered development tools
• Academic Researchers: Investigating agent-based software engineering methodologies

---

Architecture

- Hierarchical Agent Orchestration

The system implements a hierarchical process flow where agents operate in a structured sequence with decision points for quality control and iterative refinement.

┌─────────────────────────────────────────────────────────────────┐
│                    User Input (Natural Language)                │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│              InteractiveChatAgent (Loop Task)                   │
│  • Gathers requirements via conversational interface            │
│  • Uses internet_search_tool for research                       │
│  • Outputs structured requirement specification                 │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Decision Task (Quality Gate)                 │
│  Conditions: approve → planning_task                            │
│              revise → loop_interactive_task                     │
│              reject → loop_interactive_task                     │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                    PlannerAgent                                 │
│  • Analyzes requirements                                        │
│  • Researches implementation options                            │
│  • Generates JSON specification with:                           │
│    - Agent definitions                                          │
│    - Tool requirements                                          │
│    - Workflow steps                                             │
│    - Dependencies                                               │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                    CoderAgent                                   │
│  • Generates Python code from JSON specification                │
│  • Uses code_interpreter for safe execution                     │
│  • Implements: main.py, agents.py, tools.py, requirements.txt   │
│  • Tools: execute_code, analyze_code, format_code, lint_code    │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                    TesterAgent                                  │
│  • Performs static code analysis                                │
│  • Validates against original plan                              │
│  • Generates test report with confidence scores                 │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│              Test Decision Task (Quality Gate)                  │
│  Conditions: approve → deployer_task                            │
│              revise → loop_interactive_task                     │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                    DeployerAgent                                │
│  • Generates Docker/Docker Compose configurations               │
│  • Creates Kubernetes manifests (if needed)                     │
│  • Ensures localhost deployment                                 │
└─────────────────────────────────────────────────────────────────┘

---

Agent Specifications

1. InteractiveChatAgent

Assigned Role: Conversational Interface Manager
Model: gpt-4o-mini (configurable)

Available Tools:

• create_chat_interface(): Interactive terminal-based chat
• internet_search_tool(): Real-time information retrieval

Task Type: Loop (iterative conversation)
It creates a ChatInterface to guide the user for inputs that fulfills the basic requirements for the initial setup/planning to handover the requirements to the next agent. This task is reiterated until the requirements are well-defined & makes sure no field is left unfulfilled.

Output Format: Structured requirement specification with:

  - Programming language selection
  - Framework recommendations
  - Implementation type (API, Web App, Desktop, Mobile, CLI)
  - Deployment preferences

2. PlannerAgent

Assigned Role: Senior Project Manager / System Architect
Model: gpt-4o-mini (configurable)
Reasoning Steps: 3 (multi-step reasoning enabled)
Tool Used: internet_search_tool()
This agent transforms the gathered user requirements into a well-defined plan for the code implementation using the internet tool to meet with available frameworks and refers for further assistance in breaking down into details such as dependencies, required tools & functions, etc., which would be handed over to the next agent, CoderAgent.

Output Format: JSON specification containing:

{
  "project_name": "string",
  "agents_to_create": [{"class_name": "...", "description": "..."}],
  "required_tools": [{"function_name": "...", "description": "..."}],
  "workflow": ["step 1", "step 2", ...],
  "dependencies": ["package1", "package2", ...]
}

3. CoderAgent

Assigned Role: Senior Python Developer
Model: gpt-4o-mini (configurable)
Available Tools:

• code_interpreter(): E2B sandbox execution
• execute_code(): Code execution utilities
• analyze_code(): Static analysis
• format_code(): Code formatting
• lint_code(): Linting capabilities
• disassemble_code(): Bytecode analysis

By going through one by one of the above tool functions, the coding task is executed by the CoderAgent.

Output Format: Complete Python project structure:

  - `main.py`: Orchestration logic
  - `agents.py`: Agent class definitions
  - `tools.py`: Tool implementations
  - `requirements.txt`: Dependencies

4. TesterAgent

Assigned Role: Senior QA Engineer
Model: gpt-4o-mini (configurable)
Defined Function: Static code analysis and validation
Ensures the integrity of the produced code by performing test from automatic generated test cases.
Output Format: Test report JSON as:

{
  "success": boolean,
  "errors": ["error1", "error2", ...],
  "suggestions": ["suggestion1", ...],
  "confidence_score": float
}

5. DeployerAgent

To perform deployment in localhost/user machine.

Assigned Role: DevOps Engineer / Localhost Deployer
Model: gpt-4o-mini (configurable)
Available Tool: internet_search_tool()
Output Format: Deployment configurations:

  - Dockerfile
  - docker-compose.yml
  - Kubernetes manifests (optional)
  - Health check endpoints

---

System Components

- Core Framework

• PraisonAI Agents: Multi-agent orchestration framework
• Process Management: Hierarchical workflow execution
• Task Orchestration: Sequential and parallel task execution

Tools & Integrations

- Internet Search Tool

Provider: DuckDuckGo Search API
Function: internet_search_tool(query: str) -> List[Dict]
Max Results: 5 per query
Use Cases: Research, framework selection, best practices lookup
Output Format:

[
  {
    "title": "Result title",
    "url": "https://...",
    "snippet": "Result description"
  }
]

- Code Interpreter Tool

Provider: E2B Code Interpreter (sandboxed environment)
Function: code_interpreter(code: str) -> Dict
Security: Fully sandboxed execution environment
Capabilities: Python code execution, package installation, file I/O
Output Format:

{
  "results": ["output1", "output2", ...],
  "logs": {
    "stdout": ["log1", "log2", ...],
    "stderr": ["error1", ...]
  },
  "error": "error message (if any)"
}

Enterprise Features

- Caching Layer (cache/)

Files:
cache/agent_cache.py: AgentCache class with Redis integration

- Load Balancing (load_balancer/)

Files:
load_balancer/agent_balancer.py: Agent load distribution logic

- Monitoring (monitoring/)

Files:
monitoring/metrics.py: MetricsCollector class

- Security (security/)

Files:
security/key_manager.py: SecureKeyManager class
security/rate_limiter.py: Rate limiting implementation
security/validator.py: Input validation utilities

- Optimization (optimization/)

Files:
optimization/model_selector.py: CostOptimizedModelSelector class
optimization/token_optimizer.py: Token usage optimization

- Health Checks (health/)

Files:
health/health_check.py: Health check endpoints and logic

- Utilities (utils/)

Files:
utils/connection_pooling.py: Async connection pool
utils/prod_error_handler.py: Error handling decorators

- Configuration (config/)

Files:
config/production.py: Production configuration

- Logging (logging_config.py)

Logging can be enabled at runtime by setting the environment variable Debug to trueor LOGGING.LEVEL to INFO / DEBUG / ERROR / WARNING - combination of levels is possible by numerical value ( such as 30, 40 ) instead of single mentioned label.
``

- User Interface (ui/)

Files:
ui/code_analysis_ui.py: UI for Code Analysis

---

Project Structure

mas/
├── main.py                          # Main entry point and agent definitions
├── prompts.py                       # Centralized prompt templates for agents
├── model_checker.py                 # Model capability detection and selection
├── context-engineering-workflow.py # Advanced context engineering workflow
├── logging_config.py                # Production logging configuration
├── Dockerfile                       # Production container definition
├── requirements.txt                 # Python dependencies
├── README.md                        # This file
├── .gitignore                       # Git ignore patterns
│
├── cache/                           # Caching layer
│   └── agent_cache.py              # Redis + in-memory caching implementation
│
├── config/                          # Configuration management
│   └── production.py               # Production environment settings
│
├── deploy/                          # Deployment configurations
│   ├── docker-compose.yml          # Docker Compose for development
│   └── Dockerfile.dev              # Development container definition
│
├── health/                          # Health check system
│   └── health_check.py             # Health monitoring endpoints
│
├── load_balancer/                   # Load balancing
│   └── agent_balancer.py           # Agent workload distribution
│
├── monitoring/                      # Observability
│   └── metrics.py                  # Prometheus metrics collection
│
├── optimization/                    # Performance optimization
│   ├── model_selector.py           # Cost-optimized model selection
│   └── token_optimizer.py          # Token usage optimization
│
├── security/                        # Security features
│   ├── key_manager.py              # Secure API key management
│   ├── rate_limiter.py             # Request rate limiting
│   └── validator.py                # Input validation and sanitization
│
├── tests/                           # Test suite
│   ├── conftest.py                 # Pytest configuration and fixtures
│   ├── pytest.ini                  # Pytest settings
│   ├── capture_out_err.py          # Output capture utilities
│   ├── debug_utils.py              # Debugging helpers
│   ├── factories.py                # Test data factories (factory_boy)
│   ├── utils.py                    # Test utilities
│   │
│   ├── data/                       # Test data
│   │   └── test_cases.py          # Test case definitions
│   │
│   ├── integration/                # Integration tests
│   │   ├── test_multi_agent.py    # Multi-agent integration tests
│   │   └── test_tool_integration.py # Tool integration tests
│   │
│   ├── mock/                       # Mock objects
│   │   ├── llm_mock.py            # LLM API mocks
│   │   └── tool_mock.py           # Tool mocks
│   │
│   ├── performance/                # Performance tests
│   │   ├── test_load.py           # Load testing
│   │   └── test_memory.py         # Memory profiling
│   │
│   ├── unit/                       # Unit tests
│   │   ├── test_agents.py         # Agent unit tests
│   │   ├── test_tasks.py          # Task unit tests
│   │   └── test_tools.py          # Tool unit tests
│   │
│   ├── test_properties.py         # Property-based tests (Hypothesis)
│   └── test_snapshots.py          # Snapshot tests (Syrupy)
│
└── utils/                           # Utility modules
    ├── connection_pooling.py       # Async HTTP connection pooling
    └── prod_error_handler.py       # Production error handling

Key Files Description

- main.py

- context-engineering-workflow.py

- model_checker.py

---

Installation & Configuration

- Prerequisites

• Python: 3.9 or higher (3.11 recommended)
• OpenAI API Key: For OpenAI-compatible models
• E2B API Key: For code interpreter sandbox
• Redis (optional): For distributed caching
• Docker (optional): For containerized deployment

Step 1: Clone Repository

git clone https://github.com/mukundan1/MAS4MAS.git

Step 2: Create Virtual Environment

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

Step 3: Install Dependencies

pip install --upgrade pip
pip install -r requirements.txt

Step 4: Environment Configuration

Create a .env file in the project root:

# Required: LLM API Configuration
OPENAI_API_KEY=your_openai_api_key_here
OPENAI_BASE_URL=https://api.openai.com/v1  # Update for compatible APIs
OPENAI_BASE_URL=https://localhost:8000/v1  # to work with Ollama / local LLM
 
 
# Required: Code Interpreter
E2B_API_KEY=your_e2b_api_key_here
 
# Optional: Redis (for distributed caching)
REDIS_URL=redis://localhost:6379/0
 
# Optional: AWS Secrets Manager (for production key management)
AWS_REGION=us-east-1
USE_AWS_SECRETS=false
 
# Optional: Encryption (auto-generated if not provided)
ENCRYPTION_KEY=your_fernet_encryption_key_here
 
# Optional: Monitoring
PROMETHEUS_PORT=9090

Note: The OPENAI_BASE_URL should be updated according to your LLM provider:

• OpenAI: https://api.openai.com/v1
• Anthropic: https://api.anthropic.com/v1
• Custom providers: Check provider documentation

Step 5: Verify Installation

python -c "import praisonaiagents; print('Installation successful')"

---

Usage

- Basic Usage

Method 1: Direct Execution

python main.py

This will:

1. Initialize all agents
2. Start the interactive chat interface
3. Execute the hierarchical workflow
4. Generate code, tests, and deployment configurations

Method 2: PraisonAI CLI

After initializing the virtual environment, you can use the below commands:

# Initialize a new multi-agent system
praisonai --init "Create a stock analysis system with research and reporting agents"
 
# Run with specific framework
praisonai --init "Your prompt" --framework crew
praisonai --init "Your prompt" --framework autogen
 
# Execute using agents.yaml
praisonai

Advanced Usage

Custom Agent Configuration:

Modify agent definitions in main.py:

CustomAgent = Agent(
    name="CustomAgent",
    role="Custom Role",
    goal="Custom goal",
    backstory="Custom backstory",
    llm="gpt-4",  # Use different model
    api_key=api_key,
    reasoning_steps=5,  # Increase reasoning depth
    tools=[custom_tool],
    verbose=True
)

Custom Workflow:

Create custom task sequences:

custom_workflow = PraisonAIAgents(
    agents=[Agent1, Agent2, Agent3],
    tasks=[task1, task2, task3],
    process="sequential",  # or "hierarchical", "parallel"
    max_retries=5,  # retry for looping / fine-processing 
    manager_llm="gpt-4",
    verbose=True
)

Using Context Engineering:

from context_engineering_workflow import ContextEngineeringWorkflow
 
workflow = ContextEngineeringWorkflow(
    project_path="./my_project",
    llm="gpt-4o-mini"
)
 
result = workflow.execute("Create a REST API for user management")

---

Technical Specifications

Agent Communication Protocol

Agents communicate through:

1. Task Context: Shared context passed between tasks
2. Output Files: Structured markdown/JSON files
3. Decision Tasks: Conditional routing based on quality gates

Task Types

1. Standard Task: Sequential execution with context passing
2. Loop Task: Iterative execution until condition met
3. Decision Task: Conditional routing based on output evaluation

Output Formats

- Planning Output (Planning.md)

# Implementation Plan
 
## Project: [project_name]
 
## Agents
- [Agent descriptions]
 
## Tools
- [Tool specifications]
 
## Workflow
1. Step 1
2. Step 2
...
 
## Dependencies
- package1
- package2

- Code Output (Code.md)

- Generated Python files
- Code explanations
- Execution instructions

- Test Output (Test.md)

- Test results
- Confidence scores
- Error reports
- Suggestions

API Integration

- OpenAI-Compatible APIs

The system supports any OpenAI-compatible API by configuring OPENAI_BASE_URL:

# Example: Using Anthropic Claude
OPENAI_BASE_URL=https://api.anthropic.com/v1
 
# Example: Using local LLM / Ollama
OPENAI_BASE_URL=http://localhost:8000/v1

- Model Selection

The system includes intelligent model selection:

from model_checker import ModelCapabilities
 
capabilities = ModelCapabilities()
best_model = capabilities.recommend_model({
    "type": "code_generation",
    "complexity": "high",
    "context_needed": 8000
})

---

Research Contributions

Hierarchical Multi-Agent Orchestration

This system demonstrates a hierarchical process flow with quality gates, enabling:

• Iterative refinement through decision tasks
• Quality control at multiple stages
• Context preservation across agent boundaries

Context Engineering Integration

The context-engineering-workflow.py demonstrates advanced context engineering:

• Codebase analysis for context generation
• Context-enhanced agent prompts
• Validation loops for quality assurance

Enterprise-Grade Features

Production-ready features for research and deployment:

• Caching: Reduces API costs and improves latency
• Monitoring: Prometheus metrics for observability
• Security: Encrypted key management, rate limiting
• Optimization: Cost-aware model selection

Testing Framework

Comprehensive test suite including:

• Unit Tests: Agent, task, and tool isolation
• Integration Tests: Multi-agent workflows
• Performance Tests: Load and memory profiling
• Property-Based Tests: Hypothesis-based testing
• Snapshot Tests: Regression testing with Syrupy

---

Performance & Optimization

- Caching Strategy

• Local Cache: In-memory LRU cache for fast access
• Redis Cache: Distributed caching for multi-instance deployments
• TTL: Configurable time-to-live (default: 3600 seconds)
• Cache Keys: MD5 hash of agent name + input text

- Model Selection Optimization

The CostOptimizedModelSelector selects models based on:

• Task type requirements
• Cost constraints
• Capability matching

- Token Optimization

• Token counting using tiktoken
• Context window management
• Prompt optimization

- Load Balancing

• Round-robin distribution
• Least-connections algorithm
• Agent-specific routing

---

Security

API Key Management

• Encryption: Fernet symmetric encryption
• AWS Secrets Manager: Production-grade secret storage
• Environment Variables: Secure key storage

Rate Limiting

• Per-agent request throttling
• Configurable rate limits
• Sliding window algorithm

Input Validation

• Sanitization of user inputs
• SQL injection prevention
• XSS protection
• Command injection prevention

Code Execution Security

• E2B Sandbox: Fully isolated execution environment
• No Network Access: Sandboxed network isolation
• Resource Limits: CPU and memory constraints

---

Testing

- Running Tests

# Run all tests
pytest
 
# Run with coverage
pytest --cov=. --cov-report=html
 
# Run specific test categories
pytest tests/unit/          # Unit tests
pytest tests/integration/   # Integration tests
pytest tests/performance/   # Performance tests
 
# Run with verbose output
pytest -v
 
# Run with specific markers
pytest -m "not slow"        # Skip slow tests

- Test Structure

• Unit Tests: Fast, isolated component tests
• Integration Tests: Multi-agent workflow tests
• Performance Tests: Load and memory tests
• Property-Based Tests: Hypothesis-generated test cases
• Snapshot Tests: Regression testing

- Test Utilities

• Factories: factory_boy for test data generation
• Mocks: LLM and tool mocks for isolated testing
• Fixtures: Reusable test configurations

---

Deployment

- Docker Deployment

Production:

docker build -t mas:latest .
docker run -p 8000:8000 --env-file .env mas:latest

Development:

cd deploy
docker-compose up

- Kubernetes Deployment

The system can be deployed to Kubernetes with:

• ConfigMaps for configuration
• Secrets for API keys
• Deployments for agent instances
• Services for load balancing

- Health Checks

Health check endpoint: http://localhost:8000/health

Returns:

{
  "status": "healthy",
  "agents": ["agent1", "agent2", ...],
  "timestamp": "2024-01-01T00:00:00Z"
}

- Monitoring

Prometheus metrics available at: http://localhost:9090/metrics

Key metrics:

• agent_requests_total: Total requests by agent and status
• agent_request_duration_seconds: Request latency
• active_agents: Number of active agents
• token_usage_total: Token usage by model and agent

---

Contributing

Development Setup:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Install development dependencies: pip install -r requirements.txt

Optional
- 1. Run tests: `pytest`
- 2. Format code: `black .`
- 3. Lint code: `pylint *.py`

4. Commit changes: git commit -m 'Add amazing feature'
5. Push to branch: git push origin feature/amazing-feature
6. Open a Pull Request

---

License

Copyright 2025 Mukundan

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

---

Acknowledgments

Frameworks & Libraries:

• PraisonAI Agents: Multi-agent orchestration framework
• OpenAI Compatible API: Compatible API, please refer to the respective provider
• E2B Code Interpreter: Sandboxed code execution
• DuckDuckGo Search: Internet search functionality

Research & Development:

This system was developed as part of research into:

• Multi-agent systems and orchestration
• Automated software development
• AI-powered code generation
• Hierarchical agent workflows

---

Contact & Support

• Issues: GitHub Issues
• Author: Mukundan
• Repository: MAS4MAS

---

Made with ❤️ by Mukundan