In today’s digital landscape, hate speech is an escalating concern, often fueling division and unrest across communities. DweshaMukt is an Advanced Multilingual and Multimodal Hate Speech Detection System designed to counteract this issue by harnessing the power of Bidirectional Encoder Representations from Transformers (BERT) alongside cutting-edge Deep Learning and Natural Language Processing (NLP) techniques.
Our system tackles a unique challenge: detecting hate speech within Hinglish—a dynamic blend of Hindi and English—while also supporting Hindi and English languages individually. DweshaMukt leverages a pre-trained BERT model, specially optimized for real-time scenarios, offering robust analysis across a range of media. Its Multilingual and Multimodal architecture enables Hate Speech Detection across diverse content types: Text, Audios, Images, Videos, GIFs, and YouTube Comments.
With an accuracy of 88%, DweshaMukt stands as a promising solution for real-world hate speech detection applications, bridging language and media barriers to ensure safer, more inclusive online spaces.
Index Terms: Hate Speech Detection, BERT, Deep Learning, Natural Language Processing, Multilingual, Multimodal, Hinglish, Real-Time Analysis
| Team Members | GitHub Profile | LinkedIn Profile |
|---|---|---|
| Yash Suhas Shukla | GitHub | |
| Tanmay Dnyaneshwar Nigade | GitHub | |
| Suyash Vikas Khodade | GitHub | |
| Prathamesh Dilip Pimpalkar | GitHub |
| Guide | Gmail |
|---|---|
| Prof. Rajkumar Panchal | rajkumar.panchal@vpkbiet.org |
Completing this project was indeed a challenging task, and we deeply appreciate Prof. Rajkumar Panchal Sir for being our mentor. He guided us through every phase of the project, and his support was invaluable.
Hate speech detection has become a critical area of research in recent years, driven by the proliferation of social media platforms where users frequently engage in discussions that transcend linguistic boundaries. This has created unique challenges, particularly in detecting hate speech within code-switched and multilingual contexts. Numerous studies have tackled this issue by employing advanced machine learning and deep learning techniques, striving to enhance the accuracy and robustness of hate speech classifiers. This section highlights significant contributions in this field, with a focus on methodologies and outcomes that address the complexities of hate speech detection in mixed-language data.
Our model processes multiple media types for hate speech detection:
This pipeline integrates NLP and DL techniques for detecting hate speech in Hinglish, leveraging BERT for multilingual and code-mixed processing with robust performance across diverse media types.
The backend development was an intricate journey, involving months of rigorous research, experimentation, and iterative coding. Each phase contributed to refining the system’s ability to detect hate speech across various input types and languages.
Our Mark Model Index Document provides a comprehensive overview of this journey, showcasing each model’s evolution, from early concepts to the final optimized versions. Dive into the document to see how each model was crafted, tested, and fine-tuned to tackle the challenges of multilingual, multimodal hate speech detection.
The backend architecture of DweshaMukt enables the system to classify various forms of input text, audio, video, images, GIFs, and live YouTube comments by first converting each to text before applying the hate speech detection model. Here are the main scenarios handled by the system:
Google Speech to Text API, then classifies it.Google Cloud Vision API.Google Video Intelligence API for text extraction.pytchat library, then classifies them.The combined code integrates all these scenarios into a unified detection system, including added emoticon-based classification for enhanced accuracy.
The DweshaMukt Dataset is a curated collection of comments carefully selected to support research on hate speech detection. This dataset includes multilingual data in Hinglish, Hindi, and English, capturing various instances of hate and non-hate speech. It is a valuable resource for researchers and developers working on projects aimed at building safer online communities.
To ensure responsible and secure usage, access to the DweshaMukt dataset is granted upon request. Please complete the form below to submit your application. We review each request to verify alignment with our project’s objectives.
Note: Approved requests will receive an email with download instructions within 2-3 business days.
By requesting access to this dataset, you agree to the following:
The frontend for this project is built as a Streamlit application, allowing users to interact seamlessly with our hate speech detection models across various input formats. This interface makes it easy to submit and analyze text, audio, video, images, GIFs, and YouTube comments.
We utilized the following datasets:
Combined Dataset Summary:
This setup enabled efficient training and testing of the hate speech detection model with a focus on robust performance in identifying hate and non-hate content.
The performance of our model on the test set is summarized in the table below:
The overall accuracy of our model is 0.88, with a macro average of 0.59 and a weighted average of 0.88.
Model Performance
Precision and Recall:
- Hate (Yes): Precision = 0.85, Recall = 0.88
- Non-Hate (No): Precision = 0.91, Recall = 0.88
These results demonstrate the model’s robustness in identifying hate speech, making it highly effective for monitoring social media.
This study introduces a multimodal and multilingual hate speech detection model leveraging BERT and advanced NLP techniques. By integrating textual and non-textual data—including images, videos, emoticons, memes, and YouTube comments—the model significantly enhances hate speech detection across diverse contexts. Key highlights include:
The DweshaMukt project is integrated with Telegram through a series of specialized bots, each designed to handle a different type of input. This allows users to classify text, audio, images, GIFs, video, and YouTube comments directly within the Telegram platform.
| Bot Name | Description | Watch in Action |
|---|---|---|
| Haspe Text | Processes text input. |  |
| Haspe Audio | Processes audio input. |  |
| Haspe Image | Processes image input. |  |
| Haspe GIF | Processes GIF input. |  |
| Haspe Video | Processes video input. |  |
| Haspe YouTube | Processes YouTube live comments. |  |
Each bot seamlessly interacts with the backend, delivering real-time classification results to users. Whether you're analyzing text, multimedia, or live YouTube comments, these bots ensure a versatile and accessible experience for hate speech detection.
The DweshaMukt project was proudly showcased at the Nexus 1.0 State Level Project Competition on 15th April 2024. Held at the Army Institute of Technology, Pune, this prestigious event was organized by the Department of Information Technology and Computer Engineering under the AIT ACM Student Chapter.
Representing this project at Nexus 1.0 allowed our team to not only share our research and technical achievements but also to raise awareness about the importance of addressing hate speech in today’s digital world. Competitions like these offer valuable platforms for knowledge exchange, constructive feedback, and networking with other innovators, researchers, and industry experts.
Below are the participation certificates awarded to our team members for presenting DweshaMukt at Nexus 1.0.
Presenting this project at an international platform has been a milestone achievement for our team. Our research was showcased at the CVMI-2024 IEEE International Conference on Computer Vision and Machine Intelligence, hosted by IIIT Allahabad, Prayagraj on 19th and 20th October 2024. The conference offered a valuable opportunity for knowledge exchange with global experts and researchers, fostering discussions on the latest advancements in computer vision and machine learning.
The conference report details our experiences and learnings from the event, including keynote sessions and other relevant presentations on emerging research trends. To read the Conference Report, press the button below.
To view the entire official IEEE Research Paper Copy downloaded from IEEE Website, press the button below.
To read our Research Paper on IEEE Website, press the button below.
To view the entire official IEEE Research Paper Copy downloaded from IEEE Website, press the button below.
The following certificate was awarded for my participation and representation at CVMI-2024:
Securing copyright for this project marked an important milestone in safeguarding our innovation and intellectual property. Copyrighting our project not only protects the unique aspects of our hate speech detection system but also reinforces our commitment to responsible AI research. By copyrighting this idea, we ensure that the methods, models, and technological advances developed through this project remain attributed to our team.
Copyright Publication Date: 25th October 2024
Above Copyright certificate has been slightly edited to hide the personal details of the authors involved in the Copyright.
This project was generously funded by Vidya Pratishthan's Kamalnayan Bajaj Institute of Engineering and Technology College, whose support played a pivotal role in enabling our team to bring this ambitious vision to life. This funding allowed us to access essential resources, collaborate with experts, and ensure high-quality development across every phase of the project.
| Sr No | Demand Reason | Demand Cost |
|---|---|---|
| 1 | Google Colab Pro | 1025 |
| 2 | Online Courses | 2684 |
| 3 | Project Presentation Competition | 500 |
| 4 | Stationary Cost | 500 |
| Total | 4709 |
The above provided certificate is custom designed and not officially presented by the college itself. We extend our heartfelt gratitude to VPKBIET College for their trust and support. Their investment in this project has been invaluable in pushing the boundaries of AI-driven hate speech detection.
This project report is extremely detailed in terms of all the progress made till date in the project.
The project report can be viewed by pressing the button below.
[1] https://library.fiveable.me/key-terms/ap-gov/hate-speech
[2] Bansod, Pranjali Prakash, ”Hate Speech Detection in Hindi” (2023).
Master’s Projects. 1265. DOI: https://doi.org/10.31979/etd.yc74-7qas,
https://scholarworks.sjsu.edu/etd projects/1265
[3] Mohit Bhardwaj and Md Shad Akhtar and Asif Ekbal and Amitava
Das and Tanmoy Chakraborty, ”Hostility Detection Dataset in Hindi,”
in arXiv, 2020, eprint-2011.03588.
[4] P. Kapil, G. Kumari, A. Ekbal, S. Pal, A. Chatterjee and B. N.
Vinutha, ”HHSD: Hindi Hate Speech Detection Leveraging Multi-Task
Learning,” in IEEE Access, vol. 11, pp. 101460-101473, 2023, doi:
10.1109/ACCESS.2023.3312993.
[5] V. Rahul, V. Gupta, V. Sehra, and Y. R. Vardhan, ”Ensemble Based
Hinglish Hate Speech Detection,” 2021 5th International Conference on
Intelligent Computing and Control Systems (ICICCS), Madurai, India,
2021, pp. 1800 1806, doi: 10.1109/ICICCS51141.2021.9432352.
[6] H. Watanabe, M. Bouazizi, and T. Ohtsuki, ”Hate Speech on Twitter: A
Pragmatic Approach to Collect Hateful and Offensive Expressions and
Perform Hate Speech Detection,” IEEE Access, vol. 6, pp. 13825-13835,
2018, doi: 10.1109/ACCESS.2018.2806394.
[7] S. Ghosal, A. Jain, D. K. Tayal, V. G. Menon, and A. Kumar, ”Inculcating
Context for Emoji Powered Bengali Hate Speech Detection using
Extended Fuzzy SVM and Text Embedding Models,” ACM Transactions
on Asian and Low- Resource Language Information Processing,
accepted March 2023, doi: 10.1145/3589001.
[8] S. Khan et al., ”HCovBi-Caps: Hate Speech Detection Using Convolutional
and Bi-Directional Gated Recurrent Unit With Capsule Network,” IEEE Access,
vol. 10, pp. 7881-7894, 2022, doi: 10.1109/ACCESS.2022.3143799.
[9] A. K. Yadav, A. Kumar, S. ., K. ., M. Kumar, and D. Yadav,
”Hate Speech Recognition in multilingual text: Hinglish Documents,”
TechRxiv, Preprint, 2022, doi: 10.36227/techrxiv.19690177.v1.
[10] V. Rahul, V. Gupta, V. Sehra, and Y. R. Vardhan, ”Hindi-English Code
Mixed Hate Speech Detection using Character Level Embeddings,”
2021 5th International Conference on Computing Methodologies and
Communication (ICCMC), Erode, India, 2021, pp. 1112 1118, doi:
10.1109/ICCMC51019.2021.9418261.
[11] Y. Zhou, Y. Yang, H. Liu, X. Liu, and N. Savage, ”Deep Learning Based
Fusion Approach for Hate Speech Detection,” IEEE Access, vol. 8, pp.
128923-128929, 2020, doi: 10.1109/ACCESS.2020.3009244.
[12] H. Saleh, A. Alhothali, and K. Moria, ”Detection of Hate Speech
using BERT and Hate Speech Word Embedding with Deep Model,”
Applied Artificial Intelligence, vol. 37, no. 1, pp. 2166719, 2023, doi:
10.1080/08839514.2023.2166719.
[13] K. Mnassri, P. Rajapaksha, R. Farahbakhsh, and N. Crespi,
”Hate Speech and Offensive Language Detection using
an Emotion-aware Shared Encoder,” arXiv
[14] J. M. Perez, H. Saleh, A. Alhothali, and K. Moria, ”Assessing the Impact ´
of Contextual Information in Hate Speech Detection,” IEEE Access, vol.
11, pp. 30575-30590, 2023, doi: 10.1109/ACCESS.2023.3258973.
[15] https://chat.openai.com.
This project is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
By using this project, you agree to give appropriate credit, not use the material for commercial purposes without permission, and share any adaptations under the same license.
Attribution should be given as:
"DweshaMukt Project by DweshaMukt Team (https://github.com/StudiYash/DweshaMukt)"
Quick Overview regarding the permissions of usage of this project can be found on LICENSE DEED : CC BY-NC-SA 4.0
Made with ❤️ by DweshaMukt Team
