ScrapYoutuber

About

ScrapYoutuber is a time-saving tool that helps sponsors and companies quickly gather essential data about YouTubers, including main topics, social media presence, and engagement rates. Powered by the Langraph framework, it streamlines the process of evaluating YouTube channels for marketing and partnerships.

Demo

Inference Pipeline

Features

• YouTube Channel Insights: Automatically scrape main topics covered by the YouTuber.
• Social Media Extraction: Gather links to other social media accounts like Instagram, Twitter, etc.
• Engagement Metrics: Retrieve key engagement metrics such as view counts, likes, comments, and subscriber data.
• Multi-Agent System: Leverages a system of intelligent agents to distribute tasks and ensure efficient web scraping and data retrieval.
• Powered by LLMs: Uses advanced language models to process and summarize the collected information.
• Retrieval-Augmented Generation (RAG): Ensures accurate and contextually relevant data by retrieving information from multiple sources.
• YouTube API Integration: Seamlessly integrates with the YouTube Data API for additional metadata and statistics.

Technologies Used

• Multi-Agent System (LangGraph): Efficient parallel task execution.
• Web Scraping (Tavely API): Gathers information from YouTube and social media.
• Retrieval-Augmented Generation (RAG) (LangChain, Chroma, NVIDIA API, etc.): Retrieves and summarizes relevant information.
• YouTube API: Accesses structured data from YouTube.

Installation

To run ScrapYoutuber locally, follow these steps:

Clone the repository:

git clone https://github.com/ITSAIDI/ScrapYoutuber.git
cd ScrapYoutuber

Install dependencies:

pip install -r requirements.txt

Set up API keys:

• Get your API key from YouTube Data API.
• Store it in a .env file or directly in the configuration file.
• You also need to add an LLM_API_Key. The default is Fireworks API, but you can change that in LLMs.py.

Run the application:

python main.py

Usage

1. Provide the YouTuber’s channel URL to the tool.
2. ScrapYoutuber will automatically collect and display information, including:
   • Main content topics
   • Links to social media accounts
   • Key engagement metrics (e.g., average views, likes, comments)
3. Summarized results will be displayed in the terminal or saved to a file.