This project uses artificial intelligence to translate mangas or comics in different languages to any language. It also colorizes black-and-white mangas. It translates colored mangas or comics and black-and-white ones. If desired, it can also colorize. The project consists of different steps.
With a simple change in code, it can translate between different languages.
To extract text from images, OCR is used. In this project, I chose the PaddleOCR library. PaddleOCR is faster and gives better results than other OCR libraries like EasyOCR or Pytesseract, so I decided to use it in the project.
For translation, I initially planned to use the Transformers library with different language models. After testing, I did not achieve the desired results, so I decided to use the DeepL API. To use the DeepL API, register at DeepL Pro API and get your API key. Add your key in the code at line 13 where it says api = "API-KEY".
To add translated text into images, the original text needs to be removed first. For this, the simple-lama-inpainting library places boxes over the text, and AI fills the closed areas.
To colorize black-and-white images, I used the project from Manga Colorization v2. I adapted this project into my code to colorize black-and-white images. If you set colorize = 1 at line 12, it will colorize images. If colorize = 0, it will not colorize images.
Before running the project, follow these steps to create a virtual Python environment and install libraries:
Create a Virtual Environment:
python3 -m venv myenv
Activate the Virtual Environment:
source myenv/bin/activate
.\myenv\Scriptsctivate
Install Libraries:
After activating the virtual environment, install the required libraries with:
pip install deepl==1.17.0 paddleocr==2.7.3 paddlepaddle==2.6.1 simple-lama-inpainting==0.1.0 torch==2.2.2 torchvision==0.17.2 tqdm==4.66.2 textwrap3==0.9.2 pillow==9.5.0
Clone Repository:
Clone the project repository:
git clone https://github.com/koesan/Manga_Comic_Colorization_and_Translation_v2.git
Download Necessary Files:
Download the generator.zip file and place it in the networks folder of the cloned repository.
Add Manga Files:
Place the manga files you want to colorize and translate into the manga folder. Supported formats: .jpg, .jpeg, .png, .webp.
Run Project:
Activate the virtual environment and navigate to the project folder:
source myenv/bin/activate cd ./Manga_Comic_Colorization_and_Translation_v2
.\myenv\Scriptsctivate cd .\Manga_Comic_Colorization_and_Translation_v2
Start Project:
Run the project with:
python3 main.py
Results:
Translated and colorized manga pages will be saved in the result folder.
Note:
- If the value at line 12 is
0, no colorization will be applied. If it is1, colorization will be applied.- Don’t forget to add your DeepL API key at line 13.
https://github.com/koesan/Manga_Comic_Colorization_and_Translation_v2/tree/main?tab=readme-ov-file
