AI for Social Media Summaries: An Encoder-Decoder Transformer System vs ChatGPT

Abstract

In recent years, automatic text summarization has become a vital area of research due to its role in improving access to and understanding of vast information across domains. The rise of social media has intensified the need for summarization tools capable of handling user-generated content such as posts, comments, and discussions. Unlike structured texts, social media content is often informal, fragmented, context-dependent, and noisy. It frequently includes slang, abbreviations, emojis, and diverse writing styles, posing unique challenges for traditional summarization methods. While conventional approaches perform well on formal text, they often struggle to capture the nuances of online discourse. This highlights the need for specialized models that can generate coherent and context-aware summaries tailored to the characteristics of social media language. Recent advances in neural architectures, particularly Transformer-based sequence-to-sequence models, have shown promise in overcoming these challenges. These models excel at capturing long-range dependencies and contextual relationships, making them well-suited for summarizing dynamic and unstructured inputs. Despite technical progress, evaluating the quality of summaries remains difficult. Standard metrics like ROUGE may not fully reflect subjective qualities such as fluency, coherence, and semantic fidelity, which are essential for human-like summarization. This paper introduces a Transformer-based summarization system designed specifically for social media comments related to topical posts. We benchmark its performance against models like ChatGPT, assessing outputs across multiple linguistic and semantic dimensions. By combining both traditional and advanced evaluation metrics, our work provides a more holistic view of summarization quality and identifies key areas for future improvement.