Chest X-ray diagnosis models face domain generalization challenges due to cross-institutional variations in imaging protocols and scanner specifications, which degrade diagnostic accuracy on unseen domains. To address this, we propose a domain-invariant learning framework leveraging the inherent anatomical consistency of medical imaging. Our method first applies a Neighborhood-Consistent Binarization Transformation (NCBT) to convert grayscale images into topology-preserving high-dimensional binary tensors, encoding pixel intensity relationships within local neighborhoods to strip device-specific textures while retaining anatomical structures. These tensors are then reconstructed into an intermediate domain via an Intermediate Domain Style-preserving Autoencoder (IDSP-AE), decoupling structural information from domain-specific features. Crucially, our framework aligns domains without requiring target domain data during training, leveraging anatomical consistency. Experiments on four public datasets show superior generalization and improved diagnostic accuracy compared to state-of-the-art methods. The source code is available at https://github.com/LZL501/NCBT .