Building retrofitting to increase energy efficiency represents a significant opportunity for reducing global energy consumption to reverse climate change crisis. Accurate building energy performance evaluation remains critical for successful building energy retrofits. To overcome the drawbacks of existing benchmarking methods in handling noisy energy effects, stochasticity, and censored data, this paper proposes a residual-based approach for benchmarking by utilizing Bayesian regression. The utility of the proposed approach has been demonstrated by a real case application on 101 residential buildings along with the comparison against a traditional benchmarking method that is based on ordinary least squares regression. It is found that the case buildings show heterogeneous energy performance conditions. Around 57% of the buildings are energy efficient but the overall energy efficiency is slightly poor. It is also found that the traditional residual-based benchmarking tends to overestimate building energy efficiency with the average residual to be 0, when compared to the results from the new approach based on Bayesian linear regression with the average residual to be −1.14 GJ. This new benchmarking method can be used to assist building retrofitting decision makers in making more robust efficiency upgrading decisions.