Certifiedunder the Multi-Agent System module in the Mastering AI Agents program.Onboarding a new API normally means hours of manual work: reading docs, mapping endpoints, figuring out auth, hand-writing a client SDK, and hand-writing the guide that explains it, repeated almost identically for every integration. Agent-Powered API Onboarding automates that end-to-end: give it a documentation URL, and a pipeline of specialized agents researches the API's real surface, builds a dependency-ordered execution plan, dry-run tests it against the live API, generates a syntax-validated Python SDK, and writes a full integration guide, then reviews everything and returns a PASS/FAIL verdict with every artifact attached. Validated against the live Stripe API: 58 endpoints extracted, a 64-task plan executed with 0 failures, and a final PASS, all in a single run.
In an ecosystem where every product now exposes an API but few ship documentation good enough to build against quickly, developers still spend hours doing the same manual work on every integration: reading docs, mapping endpoints, figuring out auth, hand-writing a client, and hand-writing the guide that explains it all. Traditional approaches leave this entirely to the developer or lean on a single monolithic LLM call to "summarize the docs," which breaks down on anything beyond a handful of endpoints and produces unverified, often unusable output.
Agent-Powered API Onboarding (https://integrationos.streamlit.app) is a multi-agent pipeline that automates this workflow end-to-end. Given nothing but a documentation URL, it researches the API's surface, builds a dependency-ordered execution plan, dispatches that plan to specialized worker agents that dry-run test the live API, generate a working SDK, and write a full integration guide — then reviews the results and reports a pass/fail verdict with every artifact attached.
This system leverages a collaborative pipeline of specialized agents, each responsible for a distinct phase of the onboarding lifecycle. Whether it's crawling and extracting structured API data, planning validation tasks, executing live dry-run tests, generating client code, or writing documentation, these agents function under the coordination of a LangGraph state machine, with a caching layer (Remem) so repeat runs against the same API skip redundant work entirely.
Highlights:
Agent-Powered API Onboarding is built on a foundational belief: onboarding a new API should be fast, verifiable, and reproducible — not a manual afternoon of reading docs and guessing at edge cases. This philosophy drives every component of the architecture, from how research is gathered to how generated code is validated before it's ever shown to the user.
The core of the system is a LangGraph StateGraph at the Orchestrator level, coordinating a set of subgraphs and worker agents, each with its own typed state and clearly scoped responsibility. The high-level flow begins when a user submits a documentation URL. This activates a pipeline of agents, each executing a role akin to a specialized member of an integration team.
Agent workflow: Orchestrator → ResearchAgent → Planner → TaskDispatcher → {TesterWorker, SDKWorker, WriterWorker} → ReviewerAgent
| Agent | Description |
|---|---|
| Orchestrator | Validates input, sequences every other agent, aggregates errors across stages, and produces the final pipeline summary |
| Research Agent | Runs a safety guardrail, checks a Remem cache, and — on a miss — crawls and extracts structured API data from live documentation |
| Planner Agent | Converts research output into a dependency-ordered execution plan of concrete validation and generation tasks |
| Task Dispatcher | Batches tasks by dependency order and routes each to the worker that handles its tool type, running independent tasks concurrently |
| Tester Worker | Executes live dry-run validation against the real API — auth challenges, endpoint reachability, rate-limit headers, error-code plausibility, webhook event names |
| SDK Worker | Generates a complete Python client SDK from the endpoint list and validates it with ast.parse before accepting it |
| Writer Worker | Generates the markdown integration guide in batches (intro/auth section, then endpoints in fixed-size groups, then a quick-start) to avoid the structural breakdown large single-shot LLM generations are prone to |
| Reviewer Agent | Evaluates all task results, retries recoverable failures, and emits the final PASS/FAIL verdict with a full report |
Research Agent
OrchestratorState and AgentState — ensuring traceable transitions between stages. Each stage produces a tangible, inspectable output (a ResearchOutput, an ExecutionPlan, a DispatchResult) rather than passing free-text between agents.The Architecture at a Glance
Agent Pipeline: Orchestrator → ResearchAgent → Planner → TaskDispatcher → {TesterWorker, SDKWorker, WriterWorker} → ReviewerAgent, backed by Firecrawl, Gemini LLM, Remem Cache, and httpx (live API).
This agent pipeline model not only makes each stage's output independently inspectable, it also means a single stage — say, the crawler's path-filtering logic, or the writer's batching strategy — can be revised without touching the rest of the pipeline.
While the pipeline automates research, testing, code generation, and documentation, it is deliberately built to keep the human in control at the points that matter most rather than treat the run as a black box.
output/ for the user to review before it's used, rather than being silently accepted as correct. The confidence score attached to research output is a deliberate signal to the user about how much scrutiny a given run deserves.| Tool / Service | Purpose | Key Features |
|---|---|---|
| Firecrawl | Live documentation crawling | Sitemap discovery (including recursive sitemap indexes and robots.txt parsing), path-based include/exclude filtering, page-count-capped crawl jobs, retry/backoff on rate limits |
| Remem | Cross-run memory-as-a-service | Caches research profiles and execution plans per API domain; skips redundant crawling and planning on repeat runs against a known API |
Gemini (via LangChain init_chat_model) | LLM inference for research, planning, and generation | Provider-agnostic client initialization; structured output extraction for typed research and plan schemas |
| httpx-based live testers | Real dry-run API validation | Direct network calls to the target API's actual endpoints — not simulated — with timeout and error handling |
| Guardrail checks | Input and content safety | Screens the input URL and intermediate agent text for unsafe content or prompt-injection attempts before crawling or generation proceeds |
agent-powered-api-onboarding/
├── main.py # CLI entry point
├── streamlit_ui.py # Streamlit web interface
├── agents/
│ ├── Orchestrator.py # Top-level LangGraph pipeline
│ ├── ResearchAgent.py # Crawl + extract subgraph
│ ├── Planner.py # Execution plan generation
│ ├── TaskDispatcher.py # Dependency-ordered task batching
│ ├── ExecutorAgent.py # TesterWorker / SDKWorker / WriterWorker
│ └── ReviewerAgent.py # Final review and verdict
├── guardrails/ # Input/content safety checks
├── LLM/ # Model client initialization
├── schemas/ # Shared Pydantic models
├── tool/ # Crawling, parsing, validation tools
├── settings/ # Config and Remem client setup
├── tests/ # Regression tests
└── output/ # Generated SDK, guide, and task results per run
Designed with a clear separation of concerns: each agent's file is independently readable and testable, and the pipeline's stages can be re-run individually via their own compiled subgraphs.
Clone the repository and set up a virtual environment:
python -m venv .venv
.venv\Scripts\Activate.ps1 # Windows
Install dependencies:
pip install -e .
Configure environment variables in a .env file:
GOOGLE_API_KEY=your_gemini_key
MODEL_NAME=gemini-2.5-flash
FIRECRAW_API_KEY=your_firecrawl_key
REMEM_API_KEY=your_remem_key
REMEM_BASE_URL=https://api.remem.online
Run the pipeline:
python main.py --url https://docs.stripe.com/api
Generated artifacts land in output/: generated_sdk.py, integration_guide.md, and task_results.json.
▶️ Streamlit Quickstart
streamlit run streamlit_ui.py
UI highlights:
The project includes regression tests covering the core workflow components, run via:
pytest -q
📊 Results
Validated end-to-end against the live Stripe API documentation: 58 endpoints extracted with full confidence (1.0), a 64-task execution plan generated and dispatched (0 failed, 0 skipped), a syntax-valid SDK produced, and a complete integration guide written — all in a single pipeline run with a final PASS verdict.
By coordinating specialized agents around real, live-verified data — actual crawled documentation, actual dry-run HTTP requests against the target API — rather than a single model asked to "just summarize the docs," this system demonstrates that API onboarding can be both fast and verifiable, producing artifacts a developer can trust rather than ones they still have to fact-check line by line.
Threat model & trust boundaries
ast.parse) before being surfaced, rejecting output that would fail to runResearchOutput, ExecutionPlan) rather than trusted as raw model textoutput/) and the optional Remem cacheThis project is released under the MIT License. Users are free to use, modify, and distribute the code — including for commercial purposes — provided the original copyright and license notice are retained. A LICENSE file with the full text accompanies the repository.