Extending VRAT: From 3D Eye Tracking Visualization to Enabling ACT-R to Interact with Virtual Reality Environments

Cognitive architectures have been used to understand learning new tasks, forgetting, error making or navigation and mental map development. The insight cognitive architectures provide can be utilized to understand human behavior at a cognitive level. In this paper, we report on the recent developments of our toolbox VRAT which will provide a framework for designing experiments, collecting and analyzing data, and then developing cognitive models that can see and interact with the environment similar to users. The differentiating factor of our toolbox from previously developed tools is that its abilities are extended to Virtual Reality (VR). The ability to create three-dimensional visual scenes and to measure responses (i.e., gaze data, head and hand movements data) to the visual stimuli enables behavioral researchers to test hypotheses in a way and scale that were previously unfeasible. The difficulty facing the researcher is that sophisticated 3D graphics engines (e.g., Unity) have been created for game designers rather than behavioral scientists. To overcome this barrier VRAT provides a plug-and-go design to help researchers to convert their 2D experiments into VR. It also enables eye tracking and eye tracking visualization in all VR experiments. Enabling researchers to collect and analyze data more efficiently. Additionally, our tool enables (a) a straight forward transition from 2D environment design to 3D, (b) an efficient way to use data collection and visualization framework and (c) an interaction method for cognitive models to extend their capabilities and see and interact with VR environments similar to users.