Reinforcement Learning from Human Feedback (RLHF) is increasingly popular for aligning machine learning models’ outputs with human values, such as honesty, helpfulness, and harmlessness. Parameter Efficient Fine-Tuning (PEFT) has also shown promising outcomes, enabling small-sized models to produce results that are on par with huge models. In this paper, the power of RLHF and PEFT has been leveraged to fine-tune the FLAN-T5 base model on the DialogSum dataset for positive dialogue summarization tasks. It was demonstrated through quantitative and qualitative analysis that FLAN-T5 fine-tuned with PEFT and RLHF while maintaining the KL-Divergence metric to avoid hallucinations, outperforms a simple FLAN-T5 in producing positive and quality summaries. Meta’s variant of BERT—RoBERTa, as the reward model, Low-Rank Adaptation (LoRA) as the PEFT technique, and Proximal Policy Optimization (PPO) as the reinforcement algorithm were used. The mean and standard deviation of toxicity scores and the ROUGE scores were used to compare the models. The results suggest that RLHF combined with PEFT addresses challenges such as large computational requirements, catastrophic forgetting, unethical responses, and hallucinations in dialogue summarization tasks.