ARNI: an EEG-Based Model to Measure Program Comprehension

Abstract

Program comprehension is a cognitive process performed in the developers’ brain to understand source code. This cognitive process may be influenced by several factors, including the modularization level of source code and the experience level of software developers. The program comprehension is widely recognized as an error-prone and effort-consuming task. However, little has been done to measure developers’ cognitive effort to comprehend program. In addition, such influential factors are not explored at the cognitive effort level from the perspective of software developers. Additionally, some cognition models have been created to detect brain-activity indicators as well as wearable Electroencephalography (EEG) devices to support these detections. Unfortunately, they are not able to measure the cognitive effort. This work, therefore, proposes the ARNI, an EEG-Based computational model to measure program comprehension. The ARNI model was produced based on gaps found in the literature after a systematic mapping study (SMS), which reviewed 1706 studies, 12 of which were chosen as primary studies. A controlled experiment with 35 software developers was performed to evaluate the ARNI model through 350 scenarios of program comprehension. Moreover, this experiment also evaluated the effects of modularization and developers’ experience on the developers’ cognitive effort. The obtained results suggest that the ARNI model was useful to measure cognitive effort. The controlled experiment revealed that the comprehension of non-modular source code required less temporal effort (34.11%) and produced a higher correct comprehension rate (33.65%) than modular source code. The main contributions are: (1) the execution of SMS in the context studied; (2) a computational model to measure program comprehension to measure source code; (3) empirical knowledge about the effects of modularization on the developers’ cognitive effort. Finally, this work can be seen as a first step for an ambitious agenda in the area of program comprehension.