Topic and Tone in YouTube Climate Discussions

Abstract

Comments are routinely used to measure engagement on social media and to infer the effectiveness of content. This matters particularly for content with educational and social value, such as environmental sustainability. Automated analyses predominantly examine the sentiment expressed in text, which can be unrelated to a video’s message and misread engagement. We examine comments on YouTube climate-change videos using a fully automated process. Off-the-shelf natural language processing tools assign provisional labels for topic relevance (on-topic or off-topic) and for sentiment polarity (positive, neutral, or negative). Comment texts are represented by word patterns with per-video weighting adjusted to mitigate bias introduced by videos with disproportionately high comment volumes. A simple machine learning model, logistic regression, then predicts topic and sentiment. We report recognition accuracy on held-out comments and analyse the sentiment distribution within on-topic and off-topic comments. The analysis is performed separately for top-level comments that respond to the video and for replies that react to other users’ comments. For top-level comments, recognition accuracies were 0.93 for topic and 0.90 for sentiment. For replies, topic accuracy was 0.94, and sentiment accuracy was 0.88. On-topic comments showed high frequencies of both positive and negative sentiments, while off-topic comments were predominantly neutral. Among the replies, on-topic neutral sentiment was less common and positive sentiment was more common than in the top-level comments. These findings show that a fully automated, topic- and tone-aware process can reliably extract engagement patterns from YouTube comments, provided that tone (sentiment) analysis is paired with topic verification and that top-level comments and replies are analysed separately. A disproportionate number of neutral tones in off-topic threads shows that analyses that ignore the topic underestimate engagement. In on-topic comments, replies are less neutral, showing a more constructive conversation. The methodology is general and can be adapted to other domains where distinguishing genuine topical engagement from background noise is important. Ultimately, this insight can guide content creators, educators, and policymakers in evaluating and fostering meaningful online discussions around critical topics.