Abstraction and Reasoning Abilities in Artificial Intelligence Applied to Solving the ARC Prize: A Systematic Literature Review

Abstract

In recent years, the development of AI-based systems has seen a drastic increase in popularity and investment. To assess and measure specific capabilities of AI-based systems, different benchmarks have been established. AI-driven approaches tend to outperform humans on most of these benchmarks, but no AI-based system was able to surpass average human performance on the Abstraction and Reasoning Corpus (ARC) benchmark. This paper presents an extensive PRISMA-guided literature review that assesses and classifies techniques and technologies utilized by solution approaches for the ARC benchmark. 538 manuscripts are screened, resulting in an inclusion of 65 publications in the final systematic literature review. As a result, a knowledge graph consisting of review protocols of manuscripts is created, that provides further insight into classification of solution approaches. Furthermore, an estimate of possible synergies and ensemble combinations between different approaches is provided by analyzing the task-level performance of solution approaches. The estimation is conducted based on the heat-maps created using the Szymkiewicz-Simpson coefficient and the Gain coefficient.