Enhancing Text Summarization Through Reinforcement Learning in LLMs: RLHF Versus RLAIF with Human Selection Strategy (HSS)

In this exposition, we unveil the impact of modified reinforcement learning from human feedback (RLHF) and Reinforcement Learning from AI Feedback (RLAIF) on text summarization problems. The main focus is to bring the key highlights from this learning process to researchers for an easier understanding while using Large Language Models (LLMs) with RLHF for future research. We also propose RLAIF with HSS in this paper and compare it with previously proposed RLHF with Human Selection Strategy (HSS) methods in the context of text summarization. RLHF with HSS helps to select the most appropriate humans for a given and assesses the feedback given by LLMs, and then passes through a reward-based system for proper feedback for the system. RLAIF produces artificial intelligence-based results rather than human feedback, which gives similar results to RLHF with HSS, as explained in the result analysis. This technique works better considering the knowledge and interest of humans to give the rightful feedback produced by LLMs, producing better results. RLAIF with HSS provides slightly better accuracy in terms of accuracy when compared to the other techniques, as apt humans are chosen to provide the most appropriate principles for RLAIF. This work also discusses the pros, cons, and ethical implications, and the scope one has to be mindful of when working with human-based feedback integrated with artificial intelligence-based systems.