Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification
Millions of memes are created and shared every day on social media platforms. Memes are a great tool to spread humour. However, some people use it to target an individual or a group generating offensive content in a polite and sarcastic way. Lack of moderation of such memes spreads hatred and can le...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-01-01
|
| Series: | Complexity |
| Online Access: | http://dx.doi.org/10.1155/2021/5510253 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849435067554725888 |
|---|---|
| author | Apeksha Aggarwal Vibhav Sharma Anshul Trivedi Mayank Yadav Chirag Agrawal Dilbag Singh Vipul Mishra Hassène Gritli |
| author_facet | Apeksha Aggarwal Vibhav Sharma Anshul Trivedi Mayank Yadav Chirag Agrawal Dilbag Singh Vipul Mishra Hassène Gritli |
| author_sort | Apeksha Aggarwal |
| collection | DOAJ |
| description | Millions of memes are created and shared every day on social media platforms. Memes are a great tool to spread humour. However, some people use it to target an individual or a group generating offensive content in a polite and sarcastic way. Lack of moderation of such memes spreads hatred and can lead to depression like psychological conditions. Many successful studies related to analysis of language such as sentiment analysis and analysis of images such as image classification have been performed. However, most of these studies rely only upon either one of these components. As classifying meme is one problem which cannot be solved by relying upon only any one of these aspects, the present work identifies, addresses, and ensembles both the aspects for analyzing such data. In this research, we propose a solution to the problems in which the classification depends on more than one model. This paper proposes two different approaches to solve the problem of identifying hate memes. The first approach uses sentiment analysis based on image captioning and text written on the meme. The second approach is to combine features from different modalities. These approaches utilize a combination of glove, encoder-decoder, and OCR with Adamax optimizer deep learning algorithms. Facebook Challenge Hateful Meme Dataset is utilized which contains approximately 8500 meme images. Both the approaches are implemented on the live challenge competition by Facebook and predicted quite acceptable results. Both approaches are tested on the validation dataset, and results are found to be promising for both models. |
| format | Article |
| id | doaj-art-e4b72186150f4f1fae1a0f818e283923 |
| institution | Kabale University |
| issn | 1076-2787 1099-0526 |
| language | English |
| publishDate | 2021-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Complexity |
| spelling | doaj-art-e4b72186150f4f1fae1a0f818e2839232025-08-20T03:26:25ZengWileyComplexity1076-27871099-05262021-01-01202110.1155/2021/55102535510253Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme ClassificationApeksha Aggarwal0Vibhav Sharma1Anshul Trivedi2Mayank Yadav3Chirag Agrawal4Dilbag Singh5Vipul Mishra6Hassène Gritli7Department of Computer Science and Engineering, Bennett University, Greater Noida 201310, IndiaDepartment of Computer Science and Engineering, Bennett University, Greater Noida 201310, IndiaDepartment of Computer Science and Engineering, Bennett University, Greater Noida 201310, IndiaDepartment of Computer Science and Engineering, Bennett University, Greater Noida 201310, IndiaDepartment of Computer Science and Engineering, Bennett University, Greater Noida 201310, IndiaDepartment of Computer Science and Engineering, Bennett University, Greater Noida 201310, IndiaDepartment of Computer Science and Engineering, Bennett University, Greater Noida 201310, IndiaHigher Institute of Information and Communication Technologies, University of Carthage, Tunis, TunisiaMillions of memes are created and shared every day on social media platforms. Memes are a great tool to spread humour. However, some people use it to target an individual or a group generating offensive content in a polite and sarcastic way. Lack of moderation of such memes spreads hatred and can lead to depression like psychological conditions. Many successful studies related to analysis of language such as sentiment analysis and analysis of images such as image classification have been performed. However, most of these studies rely only upon either one of these components. As classifying meme is one problem which cannot be solved by relying upon only any one of these aspects, the present work identifies, addresses, and ensembles both the aspects for analyzing such data. In this research, we propose a solution to the problems in which the classification depends on more than one model. This paper proposes two different approaches to solve the problem of identifying hate memes. The first approach uses sentiment analysis based on image captioning and text written on the meme. The second approach is to combine features from different modalities. These approaches utilize a combination of glove, encoder-decoder, and OCR with Adamax optimizer deep learning algorithms. Facebook Challenge Hateful Meme Dataset is utilized which contains approximately 8500 meme images. Both the approaches are implemented on the live challenge competition by Facebook and predicted quite acceptable results. Both approaches are tested on the validation dataset, and results are found to be promising for both models.http://dx.doi.org/10.1155/2021/5510253 |
| spellingShingle | Apeksha Aggarwal Vibhav Sharma Anshul Trivedi Mayank Yadav Chirag Agrawal Dilbag Singh Vipul Mishra Hassène Gritli Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification Complexity |
| title | Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification |
| title_full | Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification |
| title_fullStr | Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification |
| title_full_unstemmed | Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification |
| title_short | Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification |
| title_sort | two way feature extraction using sequential and multimodal approach for hateful meme classification |
| url | http://dx.doi.org/10.1155/2021/5510253 |
| work_keys_str_mv | AT apekshaaggarwal twowayfeatureextractionusingsequentialandmultimodalapproachforhatefulmemeclassification AT vibhavsharma twowayfeatureextractionusingsequentialandmultimodalapproachforhatefulmemeclassification AT anshultrivedi twowayfeatureextractionusingsequentialandmultimodalapproachforhatefulmemeclassification AT mayankyadav twowayfeatureextractionusingsequentialandmultimodalapproachforhatefulmemeclassification AT chiragagrawal twowayfeatureextractionusingsequentialandmultimodalapproachforhatefulmemeclassification AT dilbagsingh twowayfeatureextractionusingsequentialandmultimodalapproachforhatefulmemeclassification AT vipulmishra twowayfeatureextractionusingsequentialandmultimodalapproachforhatefulmemeclassification AT hassenegritli twowayfeatureextractionusingsequentialandmultimodalapproachforhatefulmemeclassification |