Reinforcement learning (RL) techniques like Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO) are used to align Large Language Models (LLMs) for complex reasoning tasks. PPO has high computational costs, as compared to GRPO but GRPO exhibits instability and sensitivity to biases (such as length, difficulty, and reward noise). To address these challenges, we propose Adaptive Bias-Aware Policy Optimization (ABPO), a novel critic-free RL algorithm designed for efficiency and robustness. ABPO has several important parts, like (1) adaptive sampling and loss weighting based on prompt difficulty, (2) token-level reward normalization to avoid length bias, (3) a noise filtering policy based on reward deviation and thresholds, and (4) an adaptive Kullback-Leibler (KL) divergence penalty that keeps training stable over time. We evaluated ABPO by fine tuning the Qwen2.5-Math-1.5B model on math reasoning tasks. We implemented it by modifying the TRL GRPO trainer. Our experimental results show that ABPO improves performance over the base model on benchmarks in Math 500 (pass@1: 50.8% vs. 46.8%), AIME 2024 (pass@1: 10% vs. 3.33%), and AIME 2025 (pass@1: 3.33% vs. 0), signifying the potential of ABPO for enhancing LLM reasoning capabilities.