The EEGEyeNet dataset merges EEG data with eye-tracking technology to advance cognitive research at the intersection of brain dynamics and eye movement. By developing machine learning models to predict eye movements from EEG data, we gain insights into perceptual, attentional, and cognitive processes. However, dataset outliers can compromise model integrity and accuracy. This paper explores the impact of outliers on the state-of-the-art model and highlights the benefits of outlier removal. By identifying and eliminating outliers, we improved the dataset to enhance model performance. Through the integration of advanced modeling techniques from EEGViT and EEGViT-TCNet, we set a new standard in eye-tracking precision, reducing the RMSE from 51.8 to 48.9. Despite removing only 15 outliers out of the 21,464 total data points, we reduced the RMSE by 2.9 mm. This study underscores the critical role of data refinement in advancing Brain-Computer Interfaces (BCI) and their applications.