Searching for the smartphone lost in a house is a time-consuming task because we usually rely on a ringing sound as a target signal. To support the smartphone search lost in a house, we are developing a smartphone search assistant system that estimates the smartphone’s surrounding conditions based on acoustic sensing with a smart speaker. In this paper, we focus on smartphone contact-object estimation. Several studies have reported smartphone contact-object estimation using supervised machine learning (ML). However, the ML-based contact-object estimation fails when a smartphone is on an unknown object. There are too many objects in a house, which makes it impractical to train the object estimation model with all the objects in a house. Therefore, we propose a smartphone contact-object estimator that considers the abstraction level of estimation results to support unknown objects. Our estimator is based on two key ideas: (1) We prepare for estimator neural networks for multiple abstraction levels and switch the neural network model to a higher abstraction level when the estimation is unconfident. (2) We train the neural networks using information derived from the neural network corresponding to other abstraction levels. Experimental evaluation revealed that our proposed contact-object estimator successfully estimated a contact object with an accuracy of 0.991.