A robust point cloud registration framework should simultaneously exhibit high accuracy, computational efficiency, and strong generalizability. Achieving a balanced optimization of these three aspects remains a significant challenge, as existing methods often compromise one or more of these requirements. In this study, we introduce BUFFER, a novel registration approach designed to balance accuracy, efficiency, and generalizability. Our strategy leverages the strengths of both point-wise and patch-wise processing techniques while systematically addressing their inherent limitations. Unlike straightforward combinations of existing approaches, each module within our framework is meticulously designed to address specific challenges. Specifically, we propose a Point-wise Learner to enhance computational efficiency through keypoint prediction and to improve feature representation by estimating point orientations. This is complemented by a Patch-wise Embedder that employs a lightweight local feature extractor to generate efficient and generalized patch descriptors. Furthermore, we introduce an Inliers Generator, which integrates simple neural layers with general geometric features to identify reliable inlier correspondences. Comprehensive evaluations on real-world datasets demonstrate that BUFFER achieves an optimal trade-off between accuracy, efficiency, and generalization. Notably, our method attains state-of-the-art success rates on unseen datasets and outperforms competitive baselines by being nearly 30 times faster while maintaining superior performance.