The application of 3D object recognition of point clouds in the field of rehabilitation and daily living assistive systems is an important research area. This technology can help systems or robots to better understand and interact with their surroundings. Point cloud data can help robots perform accurate scene understanding. It can improve the robot’s autonomy and interaction, providing better service and support. However, 3D point clouds are computationally intensive for object recognition, and even more so for multi-view point clouds obtained using multiple depth cameras. In this paper, we combine the advantages of image recognition and point cloud to recognize objects using existing models of 2D image recognition and segment them in point cloud to determine the spatial coordinates. In this paper, a registered multi-view point cloud, and its 2D images obtained simultaneously at each viewpoint are used as input. Then the point cloud is preprocessed using the mask of the recognition results of the 2D image from each viewpoint to obtain the label and mask. The clustering algorithm is utilized to remove the unwanted background point cloud to obtain the desired object point cloud. This method inherits the zero-sample performance of the used 2D image algorithm, and only needs to use the original pre-training results of 2D image recognition, and does not need to be trained for point cloud recognition to find the point cloud set of the desired object and obtain the accurate spatial relationship between the objects. It is used for scene perception of intelligent systems to make them better understand their surroundings.