Rendering Involved and Machine Learning Based Environment Interpretation Integrated Geometry and Image

This research deals with 3D depth maps obtained from the environment and how to interpret aligned images. In my prior research, I rendered a simplified 3D representation, consisting of basic elements such as bricks, cylinders, and spheres, in contrast to the real-world depth map extracted from the environment. Nonetheless, when the object possesses minimal depth, detection becomes unachievable, resulting in a visual output that doesn't completely capture the true image. It is not only a visual mismatch but can not acquire the essential information for robot agents where symbols (lane, sign, signboard, etc.) drawn in indoor or urban environments coexist with humans. Accordingly, the 2D informations are traced inversely from the texture image of geometry objects that interpreted the 3D depth map. As with previous research, this study aims to bring images drawn by rendering engines as close as possible to the texture images of geometry objects with CNN-based classifiers and reinforcement learning modules, not end-to-end. Consequently, 2D geometry elements can be generated from a pixel based texture image to interpret as a limited 2D shapes. As a result of this research, not only 3D but also 2D shapes can be recognized and interpreted in our robotics agents.