Overview
AI Project Publication Assistant is a conditional multi-agent system designed to improve how AI and ML projects are presented for public sharing. While many projects are technically strong, they often suffer from unclear summaries, missing documentation sections, and poor discoverability. This system addresses that gap by analyzing a GitHub repository and producing grounded, explainable suggestions for improvement.
Unlike single-pass LLM tools, this project adopts a role-based multi-agent architecture using LangGraph. Each agent is responsible for a specific capability, enabling better control, reduced hallucination, and clearer reasoning.
---
System Scope
This system is intentionally scoped to analysis and recommendation, not execution.
It does not modify repositories, generate code, or make claims about project correctness.
Inputs
- Public GitHub repository URL
Outputs
- Suggested titles, summaries, and tags
- Documentation quality feedback
- README completeness evaluation
By keeping the scope limited to recommendations, the system remains predictable, auditable, and evaluator-friendly.
Why a Multi-Agent Approach?
A monolithic LLM workflow is insufficient for tasks that require grounding, validation, and iterative refinement. This system separates responsibilities across three agents:
- An Analysis Agent that extracts factual signals from the repository using heuristic, non-LLM tools
- A Writing Agent that proposes improved titles, summaries, and tags
- A Review Agent that evaluates output quality and enforces a bounded retry loop
This separation improves reliability, modularity, and explainability.
Orchestration Strategy
LangGraph is used to model the system as a conditional state machine. Agents communicate through shared state, and the reviewer agent acts as a quality gate. If issues are detected, the system retries generation once before terminating. This bounded retry mechanism prevents infinite loops while still allowing quality-driven refinement.
LangGraph was chosen because it enables explicit state transitions, conditional routing, and debuggable agent workflows.
This makes agent coordination transparent and avoids implicit control flow often found in monolithic agent loops.
Safety and Guardrails
Several design choices are used to reduce hallucination and uncontrolled behavior:
- Repository understanding is extracted using heuristic, non-LLM tools before generation
- The writing agent proposes suggestions rather than overwriting content
- A reviewer agent acts as an explicit quality gate
- A single bounded retry prevents infinite loops or runaway behavior
- No persistent memory or autonomous self-modification is allowed
These guardrails ensure that the system remains deterministic and explainable.
Tool Integration
The system integrates multiple tools beyond basic LLM calls, including a GitHub repository reader, RAKE-style keyword extraction, heuristic README completeness scoring, and structured output generation in YAML and Markdown formats. This tool-augmented approach improves transparency and evaluator confidence.
Outcomes and Evaluation
Each run produces deterministic, reproducible artifacts that summarize repository understanding, writing suggestions, and review feedback.
Evaluation is performed using explicit signals, including:
- README section completeness scoring
- Detection of missing or unclear documentation areas
- Heuristic checks on title length and clarity
- Reviewer-identified issues and suggested fixes
This approach prioritizes correctness and explainability over purely generative fluency.
Limitations and Future Work
Current analysis focuses primarily on README files and uses heuristic scoring. Future improvements include deeper code analysis, LLM-based semantic review, batch processing of multiple repositories, and interactive user interfaces.