Deep neural networks (DNNs), despite their impressive performance in various tasks, often produce overconfident predictions on out-of-distribution (OoD) data, which can lead to severe consequences, especially in safety-critical applications. Monitoring OoD samples for DNNs at runtime is thus essential. Although this problem has been extensively studied in image classification and recently in object detection with the Faster R-CNN architecture, the state-of-the-art series of You Only Look Once (YOLO) remains underexplored. This short paper presents an initial exploration into OoD detection for YOLO models, proposing a box-based monitor approach. The core idea is to use a data structure with a geometric shape of boxes to enclose regions in the logit space where the neural network makes decisions on in-distribution (ID) data. This structure serves as a reference to monitor the behavior of the network during deployment. Our preliminary results demonstrate that this box-based monitor outperforms several existing logits-based scoring methods, achieving a significant \(20\%\) reduction in false positive rates for OoD samples while maintaining a high true positive rate for ID samples. We hope that our work will spark meaningful discussion and inspire future research efforts, highlighting both the potential and the challenges of integrating OoD detection with the YOLO architecture for effective runtime monitoring.