The Impact of Human-AI Interaction Patterns on Problem Solving, AI Literacy, and Metacognition

Abstract

Human-AI interaction, particularly in educational contexts, is a dynamic and cognitively demanding process that holds promise for enhancing goal-directed learning. Yet, there remains a scarcity of empirical studies that examine how learners’ interaction with generative AI (GenAI) varies in structure and how these patterns influence distinct learning outcomes. This study investigates the relationship between human-AI interaction processes and outcomes such as AI literacy, problem-solving skills, metacognitive strategies, and task performance. We conducted an experimental study with 45 secondary school physics student teachers engaged in a GenAI-supported lesson plan assessment task. Using questionnaire responses, trace data, and prompt logs, we coded human-AI interaction behaviours based on self-regulation and cognitive processing levels. Through sequence clustering analysis, we identified two distinct interaction patterns. Both clusters showed significant improvement in task performance, but with divergent benefits. Cluster 1 exhibited diverse regulation processes characterized by exploratory, divergent prompting and low-level cognitive engagement in the early stages. This group showed significant gains in problem-solving skills through active idea generation and broad reflection. Cluster 2 demonstrated structured regulation behaviours, initiating interaction with deep-level cognitive processing and convergent prompting. These learners made more deliberate modifications and completed full self-regulated learning (SRL) cycles—planning, monitoring, and reflecting—which led to enhanced AI literacy and metacognitive strategy use. Our findings suggest that effective human-AI collaboration goes beyond prompt diversity; structured regulation behaviours serve as a key mediator between prompting and learning gains. GenAI served as both cognitive and metacognitive scaffolding, facilitating critical assessment and productive delegation. These results contribute to SRL theory in AI contexts and emphasize the importance of process-level analysis. Limitations include a small sample and limited prompt feature analysis. Future research should explore emotion-aware AI systems, multimodal interaction data, and the impact of task complexity on interaction dynamics. This study provides practical insights for educators and designers of AI-integrated learning systems. Specifically, it highlights the importance of tailoring AI scaffolds to different learner regulation styles: for exploratory learners, scaffolds can encourage broad idea generation and reflection, while for structured learners, scaffolds should support iterative planning and monitoring. These findings underline both opportunities and limitations of current GenAI use in classrooms, suggesting concrete directions for teacher practice and instructional design.