This paper investigates the potential of query embedding-based analysis for identifying semantically meaningful unknown objects in the task of Open-World Object Detection (OWOD). Building on the PROB model, which models objectness as a probabilistic distribution in the embedding space, we visualize and examine the spatial relationships between matched and unmatched query embeddings. We define contextual unknowns as objects that are not labeled during training but appear near known class clusters due to semantic similarity. To this end, we compare two ranking strategies for selecting top-k unmatched queries, based on objectness score and PROB score, and analyze their qualitative detection performance using dimensionality reduction techniques. The results demonstrate that the PROB score more effectively highlights meaningful unknowns, including a notable case where a previously unseen giraffe is embedded near the horse cluster, suggesting semantic alignment. These findings support the feasibility of embedding-based contextual labeling, providing a foundation for future research on automatic labeling and incremental learning in OWOD.