Abstract

Security Operations Centers (SOCs) rely on accurate interpretation of system signals, logs, and operational context to detect potential threats. Traditional single-agent LLM workflows struggle with tool grounding, evidence separation, and reasoning reliability.

This project presents Cybersecurity Monitor, a modular multi-agent system that simulates a SOC-style analysis pipeline. Multiple agents collaborate through a structured orchestration layer to collect evidence, analyze security signals, and generate a grounded final security report. The system uses Model Context Protocol (MCP) for tool integration, LangGraph for agent orchestration, and DeepEval for formal evaluation of reasoning quality.

Problem Motivation

Security analysis is inherently multi-step and role-driven:

• Evidence collection must be separated from interpretation
• Analysis should avoid inventing events when data is incomplete
• Final reports must clearly communicate uncertainty

Single-agent workflows often blur these boundaries, increasing hallucination risk. This project demonstrates how a multi-agent architecture with explicit role separation improves reliability, transparency, and extensibility in security analysis tasks.

System Overview

The Cybersecurity Monitor system is composed of four primary layers:

1. Tool Layer (MCP Server)
2. Agent Layer (Specialized SOC agents)
3. Orchestration Layer (LangGraph)
4. Evaluation Layer (DeepEval)

Each layer is loosely coupled, allowing independent extension and testing.

Architecture

Tool Layer – MCP Server

The MCP server exposes controlled system-level tools to agents, including:

• System log access
• Process and port inspection
• File and environment signal checks

All tools return structured outputs and act as the only source of ground truth. Agents are not allowed to reason beyond tool-provided data.

Agent Layer – SOC Agents

The system consists of at least three agents, each with a distinct role:

Evidence Collection Agent

• Calls MCP tools
• Retrieves raw system signals
• Performs no interpretation or judgment

Security Analysis Agent

• Analyzes retrieved evidence
• Identifies potential indicators of compromise
• Explicitly handles missing or insufficient data

SOC Report Agent

• Synthesizes a final security report
• States conclusions, uncertainty, and recommendations clearly
• Mimics real SOC reporting standards

This separation enforces clean reasoning boundaries and reduces hallucination.

Orchestration Layer – LangGraph

LangGraph is used to define:

• Agent nodes
• Execution order
• Shared state between agents
• Deterministic multi-step workflows

Typical execution flow:

Evidence Collection → Security Analysis → SOC Report

This orchestration design allows easy insertion of:

• Additional agents
• Human-in-the-loop review steps
• Parallel or branching workflows

Tool Integration via MCP

The system integrates multiple tools using the Model Context Protocol:

• Log scanning tools
• Process and port inspection
• File integrity and environment checks

Each tool:

• Is invoked explicitly by agents
• Produces structured outputs
• Prevents uncontrolled model reasoning

This approach makes the system extensible and secure while maintaining strict evidence grounding.

Evaluation and Quality Assurance

To ensure reliable reasoning, the project includes an automated evaluation layer using DeepEval.

Metrics Used

• HallucinationMetric
  Detects fabricated or unsupported security claims

Tests are executed using pytest and invoke the full agent pipeline end-to-end.

Error Handling and System Resilience

The system includes multiple safeguards to ensure robustness:

• Agent Failures: If an agent fails to respond, the orchestration layer retries with exponential backoff.
• Fallback Strategies: Critical tasks (e.g., evidence parsing) have backup heuristic agents.
• Unexpected Inputs: All tool outputs are validated against schemas to prevent malformed data propagation.
• System Recovery: Containerized deployment ensures automatic restarts on crash, with logs persisted for post-mortem analysis.
• Human-in-the-Loop: Analysts can override or validate agent outputs for high-risk findings.

Example Output

A typical system output follows SOC reporting conventions:

=== FINAL SECURITY REPORT ===

Event Analysis:
No confirmed malicious activity detected based on available system signals.

Indicators of Attack:
None identified.

Assessment:
Current evidence is insufficient to assert compromise.

Recommendations:
- Continue monitoring system logs
- Perform scheduled security scans
- Review network traffic periodically

Licensing and Usage Rights

This project is released under the MIT License.
Users are free to use, modify, and distribute the code with proper attribution.
The authors assume no liability for misuse in production environments.
For full terms, see the LICENSE file in the repository.

Future Enhancements

The architecture is designed for incremental expansion. Planned future enhancements include:

• Integration of Nmap for active network scanning
• Metasploit for controlled vulnerability validation
• Wireshark or PCAP analysis for network-level inspection
• Human-in-the-loop validation for high-risk findings
• Continuous benchmarking against baseline agent workflows

All future tools will be added as new MCP endpoints without changes to agent logic.

Reproducibility and Setup

The repository includes:

• Clear setup instructions
• Environment variable management via .env and .env_example
• Fully reproducible local execution
• Automated evaluation tests

Detailed instructions are provided in setup.md and usage.md.

Conclusion

Cybersecurity Monitor demonstrates how multi-agent collaboration, tool grounding, and structured orchestration lead to more reliable security analysis than single-agent workflows. By combining MCP, LangGraph, and formal evaluation, this project provides a strong foundation for scalable and extensible SOC automation systems.