Development of an Intelligent System for Processing Semistructured Data: Industry Structuring and Advanced Analysis of Information Extracted from Comments to Video Clips in Social Networks
Scientific relevance of the study. In the era of rapidly increasing volumes of data generated by social media users, analyzing textual data such as comments is becoming one of the key challenges of modern science. Comments are a valuable source of information, allowing us to identify public sentimen...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Plekhanov Russian University of Economics
2025-05-01
|
| Series: | Открытое образование (Москва) |
| Subjects: | |
| Online Access: | https://openedu.rea.ru/jour/article/view/1073 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Scientific relevance of the study. In the era of rapidly increasing volumes of data generated by social media users, analyzing textual data such as comments is becoming one of the key challenges of modern science. Comments are a valuable source of information, allowing us to identify public sentiment, analyze users’ opinions, and track social trends. However, due to the semistructured or completely unstructured nature of these data, their processing requires innovative approaches. Purpose of research. The aim of this research is to develop an intelligent system for processing semistructured data from comments on social media videos using structuring algorithms targeting different industries. The research aims to create an efficient method to analyze tone, clustering and extract key themes from comments in order to evaluate the impact of video content on the audience. The research will propose an approach to automatically extract and structure data by industry, which will allow for a more accurate and in-depth analysis of content perception and its impact on different social and professional domains. Methods. Developing an intelligent system for analyzing semistructured data requires innovative methods and approaches that combine natural language processing (NLP), machine learning algorithms and big data analytics techniques. These methods include: automatic data extraction via API, preprocessing adapted for three languages (French, English and Russian), deep sentiment analysis using the Bert product and a probabilistic algorithm for statistical calculations, and clustering using K-Means, DBSCAN and Agglomerative algorithms. The materials are based on comments from social networks (TikTok, Instagram, Twitter, Facebook, YouTube, Reddit, VKontakte) in Russian, English and French. SpaCy and NLTK libraries were used for preprocessing, and the Hugging Face Transformers model worked with pre-trained models for sentiment analysis. Machine learning techniques including clustering and natural language processing were used. Data was structured using topic modeling and language models implemented using Python libraries. The results of the study. The development of an intelligent system for processing semistructured data has improved the analysis of comments on videos in social networks through a combination of various machine learning models and algorithms. The results of the study allowed us to develop a prototype of a comment analysis tool that effectively collects and structures data from various social networks. This data structuring led to better organization and increased accessibility of information, facilitating its utilization. By using natural language processing (NLP) methods, we identified key themes and emotions in the comments while conducting sentiment analysis that highlights major emotional trends. Clustering methods, such as K-means, grouped the comments by similar themes. Additionally, we created visualizations that show sentiment distribution, allowing users to quickly interpret the data. The integration of visualization techniques transforms complex analytical results into intuitive graphs, making it easier to understand user interactions with the content. Thus, our system proves effective in providing valuable insights and optimizing audience interaction strategies. Conclusion. The results of the study showed that the proposed approach significantly improves the accuracy of classification and structuring of semistructured data, especially when it comes to comments extracted from social media videos. The developed system uses natural language processing algorithms to analyze the data with respect to its industry, which allows for automatic structuring of comments depending on their content and detailed tone analysis. The effectiveness of this approach was validated by analyzing comments from various social platforms, which demonstrated its ability to extract and structure relevant information, as well as assess the impact of videos through user reactions. |
|---|---|
| ISSN: | 1818-4243 2079-5939 |