<p>Online order fulfillment under limited capacity is essential for e-commerce when faced with demand fluctuations or surges. This paper investigates a multi-period integrated production and delivery scheduling problem for fulfilling online orders with due dates. We model the problem as the Markov decision process and develop an approximate dynamic programming (ADP) algorithm to minimize the weighted sum of average waiting time and order timeouts. The performance and robustness of the ADP strategy are validated through numerical experiments across various scenarios with differing order volumes and demand patterns. We find that for a fixed total volume of orders, different patterns of demand arrival influence the integrated production and delivery schedules, among which scenarios with a decreasing demand pattern over time are particularly challenging. Additionally, we discover that settings with flexible delivery departure times are more advantageous when demand exceeds the predefined production capacities of businesses, whereas fixed delivery departure times do not result in significant negative impacts when demand is relative low. Our findings suggest valuable insights for e-commerce operations regarding adaptable production and delivery schedules in response to varying demand patterns.</p>