This paper presents a novel system for creating and displaying a 3D avatar model on a fan hologram device. The system uses a webcam to capture the gestures and movements of the user and applies an AI model to recognize the body parts and joints of the user. The system then maps the recognized joints to a 3D avatar model and renders it on a fan hologram device using a persistence of vision (POV) technique. The system allows the user to interact with the 3D avatar model in real time and see it from different angles. The system also provides various functions, such as changing the model, using the input as image or video, and displaying the skeleton of the user. The system is based on the mediapipe holistic solution, which combines the pose, face, and hand landmarkers to create a complete landmarker for the human body. The system is tested and evaluated in terms of basic functions, display quality, and user experience. The results show that the system performs well in most conditions and provides an immersive and engaging experience for the user. The accuracy of the recognition function varies from 55 to 100%, depending on the environment and lighting condition. The success rate of the identification function is 82.69% for all body parts and acts. The system has a high potential for further development and application in various domains, such as education, entertainment, art, and communication.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

3D Avatar Interactive System on Hologram Fan Projector Using Motion Recognition Control by Mediapipe Holistic

  • Thai Anh Ngoc Huynh,
  • Hoang Nguyen Pham,
  • Ngoc Tho Thi Nguyen,
  • Tri Thanh Trang

摘要

This paper presents a novel system for creating and displaying a 3D avatar model on a fan hologram device. The system uses a webcam to capture the gestures and movements of the user and applies an AI model to recognize the body parts and joints of the user. The system then maps the recognized joints to a 3D avatar model and renders it on a fan hologram device using a persistence of vision (POV) technique. The system allows the user to interact with the 3D avatar model in real time and see it from different angles. The system also provides various functions, such as changing the model, using the input as image or video, and displaying the skeleton of the user. The system is based on the mediapipe holistic solution, which combines the pose, face, and hand landmarkers to create a complete landmarker for the human body. The system is tested and evaluated in terms of basic functions, display quality, and user experience. The results show that the system performs well in most conditions and provides an immersive and engaging experience for the user. The accuracy of the recognition function varies from 55 to 100%, depending on the environment and lighting condition. The success rate of the identification function is 82.69% for all body parts and acts. The system has a high potential for further development and application in various domains, such as education, entertainment, art, and communication.