This research paper introduces a method to address the problem of station keeping controlling of underactuated stratospheric airships when faced with unpredictable external wind disturbances. The proposed algorithm combines weather optimal control theory and deep reinforcement learning theory to achieve station keeping control. In terms of details, an outer-loop guidance law is developed using weather optimal control theory. This guidance law allows the airship to autonomously follow a pendulum-like arc trajectory towards a stable point that counteracts the wind. Additionally, a deep reinforcement learning agent is used as the inner-loop attitude controller for the airship. A reward function is designed to facilitate model-free, self-learning control, eliminating the need for a physical model of the airship. The algorithm is finally validated through simulations, which demonstrate its effectiveness.