The design of a stope layout is the initial step in an underground stope project and helps with long term planning while also defining the mineable ore reserves and the economic value to be extracted during the life-of-mine. Decision-making at this stage is subject to uncertainties in the geological and mineral grade estimated models due to the scarce data typically available, which has a major impact on the performance of mining assets. Although geostatistical techniques have been established to characterize uncertainty with multiple conditional simulated models, incorporating these orebody simulations into the mining design process remains a major challenge. Simulated models are typically used for evaluation of deterministic mine designs rather than optimization. In this paper, a Proximal Policy Optimization (PPO) algorithm, a Deep Reinforcement Learning technique, is proposed to integrate multiple conditional orebody simulations to generate an underground stope layout that maximizes the expected profit of the mine over an ensemble of geostatistical realizations. A risk-discounted approach is then presented that penalizes the downside risk of stopes to generate a design that has a lower risk. The user controls their attitude towards risk, allowing for a risk-averse (lower profit variance over the ensemble) or opportunity-seeking (higher average profit over the ensemble) set of stopes. A case study is presented based on a gold deposit where 100 conditional simulations are generated and multiple stope designs are compared. The proposed framework achieves a higher expected value than the widely used deterministic underground stope design tool, as it uses information across all the simulated scenarios to guide the mine design process. Moreover, the proposed approach allows the practitioner to tune the risk profile of the generated stopes to align with their attitude towards profit uncertainty.