This project presents a production ready multi-agent system that evaluates a candidate's public GitHub profile against a provided job description (JD). Leveraging LangGraph for agent orchestration, the system automates the extraction of required skills from JDs, analyzes GitHub repositories for tech stack compatibility and activity, and generates a comprehensive evaluation report. The system extends earlier prototypes by enhanced reliability safeguards, automated testing, and a graphical user interface.
Clear Purpose and Objectives
The primary objective of this project is to design and implement reliable, production ready application that automates the evaluation of a candidate’s public GitHub profile against a job description (JD) using a secure, robust, and explainable multi-agent system. The system aims to streamline technical hiring by providing objective, reproducible, and scalable candidate assessments by public GitHub contributions.
The goal is to test and evaluate agentic systems, protect them from security and safety risks, deploy them as real services, and operate them over time.
Target Audience Definition
Technical recruiters and hiring managers
Engineering and AI/ML teams
HR technology solution providers
Researchers in AI-driven talent evaluation
Problem Definition
Manual screening of open-source contributions is time-consuming, subjective, and error-prone. There is a need for an automated, transparent, and secure solution to match candidate skills and activity with job requirements.
It will make the candidate assessment easy and save bunch of time.
Current State Gap Identification
Lack of automation in JD-to-GitHub profile matching
Insufficient security and validation in existing tools
Limited explainability and reproducibility in candidate evaluation
Absence of robust error handling and monitoring in demo systems
Context Establishment
This work builds on recent advances in multi-agent orchestration, LLM integration, and secure AI system design. It addresses the need for production-grade, enterprise-ready solutions in technical hiring.
Architecture
The system is composed of four main agents, each responsible for a distinct stage in the evaluation pipeline:
Agent Name
Role & Functionality
JD Analyzer Agent
Extracts programming languages and skills from the job description using LLM with static analysis fallback.
Repo Match Agent
Matches candidate's GitHub repositories to required skills.
Activity Agent
Analyzes commit activity in relevant repositories over the past year.
Evaluation Agent
Scores the match and generates a human-readable evaluation report.
Key Improvements:
SafetyValidator: Detects toxicity in JD
Retry Utility: Robust retry with exponential backoff for all network calls
Logger: Centralized, structured logging for compliance and debugging
Workflow and Processing Steps:
User provides job description (default in data/job-description.txt) and candidate GitHub username (UI/CLI)
The application validates JD & GitHub name and does sanitization.
Multi-agent workflow orchestrated (LangGraph) which includes the following steps:
Extract programming languages from JD
Check candidate GitHub Repos related to extracted programming languages
Check commit activity over the past year and advance criteria (stars and issues)
Calculate the score based on relevant repos, commit activity, advanced criteria and share the result
Output Result is also validated and presented to user
UI not loading: Ensure dependencies and port availability
Support:
GitHub Issues for bug reports and feature requests
Significance and Implications of Work
This project demonstrates how to move from a research demo to a production-ready, secure, and explainable AI system for technical hiring. It sets a new standard for automation, safety, and transparency in candidate evaluation.
Handles large GitHub profiles and complex JDs efficiently. Number of repos to fetch are configurable and tested with 50 repos.
Workflow step timings and throughput tested in integration suite
Designed for reliability under API/network failures. All steps are serial for a candidate for better production control.
Based on the current criteria, the accuracy is greater than 80%. More detailed individual agent performance analysis can be found in previous publication.
Maintenance and Support Status
This project is maintained as part of the ReadyTensor Certification Program and is intended as an educational and reference implementation of a multi-agent system.
Maintenance Status: Actively maintained for learning, experimentation, and certification purposes. Support: Community-driven. Issues and pull requests are welcome through the GitHub repository.
Access and Availability Status
Open-source, public repository
Docker image build instructions provided
Future Work
Use of GitHub profile data for better scoring. Currently, using only repo and commit information.
Use of quality (by checking README files of the top N relevant repositories) in scoring
Account repo title, description against the JD for the relevancy
In short, you are free to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software, provided that the original MIT license notice and copyright notice are included in all copies or substantial portions of the Software. The license provides the software “as is,” without warranty or liability for damages.
Certifiedunder the Production-Grade Agentic System module in the Agentic AI In Production program.
