Image Captioning and Text-to-Image with BLIP and Stable Diffusion

Objectives

• Enhance visual understanding through automated image descriptions.
• Generate high-quality images from text descriptions.
• Support multilingual captioning and speech synthesis for accessibility.

Pipeline Implementation

1. Load Models:

   • BLIP: Used for image captioning.
   • Stable Diffusion: Generates images from text.
   • Google Translator: Translates captions to multiple languages.
   • gTTS: Converts text into speech.

2. Image Captioning:

   • User uploads an image.
   • Caption is generated using BLIP.
   • Caption is translated into multiple languages.
   • Speech synthesis converts the caption to audio.

3. Text-to-Image Generation:

   • User provides a text description.
   • Stable Diffusion generates the corresponding image.

Resources

• GitHub Repository: Image-Captioning-Text2Image

• Hugging Face Space: Image-Captioning-Text2Image