This paper presents a deep reinforcement learning-based motion planning approach that uses the semantics (e.g., material or element type) of building elements to avoid areas near unsafe elements (e.g., doors that can open), referred to as risky zones. To this end, a costmap containing the risky zones is automatically generated by querying the semantics and geometry of these elements from the digital twin of a building. The robot is discouraged from entering these risky zones by defining a penalty within the reward function of a Soft Actor-Critic (SAC) deep reinforcement learning agent. The approach is validated in simulation by navigating three different environments. A comparison is made between a SAC agent trained with the costmap and one that is not. Results show that the agent using the costmap successfully avoids the risky zones in front of the doors. This research is a step towards automatically interpreting semantic elements stored in building digital twins for robot navigation.