<p>Mixed integer linear programs (MILPs) are widely used to model large-scale or time-sensitive real-world optimization problems, where high-performance computing (HPC) techniques are often required. Modern solvers rely on the branch-and-bound (B&amp;B) search algorithm, where branching decisions critically determine search efficiency. Traditional branching heuristics rely on mathematical computations at each search step, which can be very expensive when high-quality decisions are required. This has motivated learning-to-branch approaches to shift computational burden from online search to offline training. Many existing methods adopt imitation learning (IL) to mimic traditional heuristics, but incur high offline costs for expert data generation and remain fundamentally limited by expert quality. Reinforcement learning (RL) offers a promising alternative, yet learning effective branching policies from scratch is difficult and still results in prohibitively expensive offline cost. Therefore, this work proposes PEBSI, an efficient RL-based branching policy that realizes this computational burden shift while addressing the cost of offline training. PEBSI learns without expert demonstrations or restrictive solver settings. It is guided by a decision-aware reward that provides informative learning signals and a quality-aware exploration strategy that improves training sample quality. Additionally, it employs a fully parallelizable training scheme that leverages HPC resources for scalable data generation and efficient policy learning. Extensive experiments on diverse MILP benchmarks show that PEBSI consistently outperforms RL baselines trained without expert guidance and, in several settings, matches or surpasses the IL-based counterpart.</p>