The novel design of a bio-inspired aircraft wing based on the Nankeen Kestrel requires a new design for servo-level flight control. The bio-inspired design has unknown nonlinear dynamics that cannot be accurately simulated, especially in turbulent wind flow, making it an ideal platform on which to study online only learning methods. We conduct a novel investigation into online Deep Reinforcement Learning design requirements for training a closed-loop control system without simulation, including reward function shaping, state space, and action space configurations. We derive a nonlinear policy for control and turbulence mitigation of a half-wing robotic replica of a Kestrel in online real-world wind-tunnel experiments. We evaluate two DRL algorithms Twin-Delayed Deep Deterministic, and Soft-Actor-Critic. Following aerospace engineering practice, we show viable control properties are achieved on a half-wing model, which is required before full aircraft models are developed. Our work provides valuable insights into online learning of stable flight controllers for novel aircraft designs.