In this paper, an object searching and adaptive grasping system for household assisting robot is proposed, using supervised object detection to enable a robot to move to search for the target based on voice commands, grasp the specified object, and return it to the user. The proposed system includes voice interaction, autonomous motion control, object detection, and object grasping modules. For voice recognition, the Google Speech Recognition API is used to recognize and convert user speech into text. Autonomous motion control is achieved using a 3D LiDAR sensor for accurate navigation to the destination. Object detection utilizes YOLACT, an instance segmentation technique based on deep learning. It acquires the contour and distance information of objects to generate the optimal grasping posture for the robot arm. In experiments, the robot successfully understood user’s instruction, searched the ordered object in the environment and took it back to the user, showing the effectiveness of the proposed system. In the future, the robot needs to deal with different objects based on their properties and ambiguous instructions from different users.