Using ChatGPT for Quantitative Content Analysis: Opportunities and Challenges in Construction and Sustainability Research

Abstract

Artificial Intelligence (AI), especially Large Language Models (LLMs) like ChatGPT, are changing the way researchers can process and analyse qualitative data. In this paper the use of ChatGPT is tested for Quantitative Content Analysis (QCA) by applying it to interview material about digital construction technology and sustainability. Two versions of the same data are compared: (1) complete transcripts of five interviews with professors, and (2) a shorter summarized version of the same interviews (the summaries were prepared by researcher). With the same workflow, ChatGPT did several steps: preprocessing (splitting the text into words, removing very common small words, and reducing words to their basic form), keyword extraction, thematic coding with five categories, and also a simple sentiment analysis. The aims were: (a) to see if ChatGPT can find the main themes in a reliable way, (b) to compare results from full transcripts versus summaries, and (c) to understand what practical advantages and problems appear when undertaking ChatGPT in a real research situation. The results were similar at the general level: Digital Technology and Sustainability were the strongest themes in both datasets, followed by Education/Training, Benefits, and Barriers. The sentiment analysis gave slightly positive values in both (+0.18 for transcripts, +0.16 for summaries). At a more detailed level, the transcripts included more technical words (for example “embodied carbon”, “Life cycle Analysis (LCA)” and standards), while the summaries included more general terms, which made the counts higher. Some practical issues also influenced the work: undertaking a free ChatGPT account caused interruptions, sometimes the tool changed its output style, and it was difficult to export charts or tables, these problems reduced reproducibility. In conclusion, ChatGPT can be useful for first steps in QCA and for saving time in early coding, but it is not enough for final or very detailed analysis. For better use, the following suggestions are provided: a combination of AI with human checking, making domain-specific dictionaries, undertaking clear and repeated prompts, and working with more stable or professional access. This study shows both the opportunities and the real problems when ChatGPT is used for content analysis in construction and sustainability research.