This project presents a multi-agent system built with LangGraph for automated analysis of GitHub repositories. The system breaks down repository evaluation into smaller tasks handled by specialized agents, such as content extraction, metadata analysis, structure checking, and quality scoring. These agents work together in a structured workflow to process a repository’s README file and generate clear, structured feedback. The final output includes a summary of the project, extracted metadata, a quality score, and a pass/retry decision based on predefined rules. The goal is to show how agent-based systems can be used to improve clarity, structure, and consistency in automated codebase analysis.

🏷️ Tags:

📘 Introduction

README files are one of the most important parts of any GitHub repository, but their quality varies a lot across projects. Some are well structured and informative, while others are incomplete or unclear. This makes it difficult to quickly understand what a project does or how usable it is.

This project explores a multi-agent approach to solving that problem. Instead of using a single model to handle everything, the system splits the task into smaller responsibilities handled by different agents. Each agent focuses on one aspect of the repository, such as extracting information, analyzing structure, or evaluating quality.

LangGraph is used to coordinate these agents into a workflow where each step builds on the previous one. This makes the system easier to extend, debug, and reason about compared to a single monolithic model.

⚙️ Methodology

The system takes a GitHub repository URL as input and processes it through a series of agents. Each agent has a specific role and contributes to a shared state that is passed through the workflow.

First, the Analyzer Agent fetches the repository data using the GitHub API, including the README file and basic metadata like stars, forks, and programming language. This forms the base input for the rest of the system.

The workflow then splits into multiple parallel agents. The Content Agent extracts the project title and generates a short summary of the README. The Metadata Agent identifies keywords and generates relevant tags. The Structure Agent checks whether key sections like installation, usage, and license are present.

After that, the Quality Agent evaluates the README based on simple rules such as length, formatting, and the presence of code blocks. All these outputs are then passed to the Reviewer Agent.

The Reviewer Agent combines everything and produces the final decision. If the quality score is too low or important sections are missing, the system can trigger a retry loop for re-evaluation.

🏗️ Architecture

GitHub Repo URL
        ↓
┌─────────────────────┐
│   Analyzer Agent     │  → Fetch README + Repo metadata (API tool)
└─────────────────────┘
        ↓
────────────────────────────────────────
↓                ↓                ↓
Content        Metadata       Structure
Agent          Agent           Agent
↓                ↓                ↓
────────────────────────────────────────
        ↓
   Quality Agent
        ↓
   Reviewer Agent
        ↓
     Final Output

🧪 Experiments

The system was evaluated using a set of GitHub repositories with varying levels of documentation quality. The objective was to assess how well the multi-agent pipeline handles differences in structure, completeness, and clarity of README files.

The test set included:

• Well-documented repositories with clearly defined sections
• Moderately documented repositories with partial structure
• Poorly documented repositories with minimal or unstructured content

📊 Results

The system returns a structured breakdown of a GitHub repository after processing its README and metadata.

Each output includes:

• Basic repository details (stars, forks, language)
• Extracted title and short summary
• Generated tags
• Missing sections (if any)
• A quality score
• Final status (pass or retry)

🔹What the outputs show

Repositories with clear and complete READMEs usually return:

• Better summaries
• No missing sections
• Higher quality scores
• A pass result

Repositories with partial documentation:

• Have some missing sections
• Get moderate scores
• Include feedback in the output
• Poorly documented repositories:
• Produce weaker summaries
• Miss multiple key sections
• Get lower scores
• Often return a retry status

🔹 Consistency

The outputs are consistent across runs because the evaluation is rule-based.
The same input produces the same result, which makes the system predictable and easier to debug.

🔹 Traceability

Each part of the result comes from a specific agent:

• Missing sections → Structure Agent
• Score → Quality Agent
• Summary → Content Agent

🏁 Conclusion

This project shows how a GitHub repository can be broken down and understood using a simple multi-agent setup instead of a single model doing everything at once. Each agent handles a specific part of the job, reading the README, pulling metadata, checking structure, and scoring quality; then everything is combined into one final result.

The main benefit of this approach is clarity. It’s easy to see what each part of the system is doing and why a repository gets a certain score or status. It also makes the system easier to adjust or improve without breaking everything.

From the results, well-documented repositories are handled smoothly, while weaker ones are clearly flagged for missing information or poor structure. The final output stays consistent and easy to interpret across different cases.

Overall, the system works well as a lightweight way to evaluate repository quality and shows how splitting a task into smaller agents can make the process more organised and practical.