<p>This paper presents two novel reinforcement learning (RL) architectures tailored for “online”, i.e. dynamic, and ”offline”, i.e. tabular, decision support schemas. Uniqueness of the approaches stems from reliance on Operations Research (OR) methods as building-blocks, rather than data sets or actor-critique. Along with benchmark results with fast multi-criteria decision-making heuristic and the standard, i.e. greedy Q-learning schema, outcomes of investigations which aim at answering questions regarding the effect of state space dimensions or size of the control set in problem formulation are also provided. In addition to explaining the theoretical basis of the novel architectures reaching to the rudiments of dynamic programming, the article seeks to exhibit the potential of the proposed methods from the engineering perspective through designing the methods into a practical decision support unit (DSU) to be used in food planning by end-consumer. In this vein, the test framework aims to prevent food waste from household and restaurant kitchens via the DSU which serves to generate menu recommendations in a volatile market while considering three important aspects of food management: the total shopping budget, wasted ingredients, and the nutrition intake resultant by the selected menu. The results reveal the holistic superiority of online architecture which is based on Monte Carlo simulation and mathematical modelling integrated into the Bellman equation. Moreover, taking advantage of wise reduction opportunities of control set or dimension of state space are supported by the results. In addition to the solution quality assessment of each approach, implementation details such as computational requirements and resilience capacity are also discussed from the business perspective.</p>