Sylvester Hadamard matrices are frequently encountered in numerous applications. On the other hand, Yates matrices have been extensively used in statistical designs. In this paper, we highlight several key properties that characterize them, focusing on their base construction and sign spectra. Consequently, we provide a direct proof of the equivalence between Sylvester Hadamard and Yates matrices, relying solely in the equivalence of their bases. Furthermore, two properties that uniquely characterize Sylvester Hadamard matrices, the full sign spectrum property and the closure under addition/multiplication. Finally, an application to a linear regression model is also presented.