For a prime number \(p\ge 3\) , let A be a diagonal expanding matrix, and let \(0\in {\mathcal {B}}\subset {\mathbb {Z}}^2\) be a p-element digit set satisfying \(\mathcal {Z}(\hat{\delta }_{{\mathcal {B}}})=\bigcup _{j=1}^{p-1} \left( \frac{j}{p}(\omega ,\rho )^t+{\mathbb {Z}}^2\right) \) for some \(\{\rho ,\omega \}\subset \{1,\cdots ,p-1\}\) , where \(\gcd (\rho ,\omega )=1\) . The associated self-affine measure \(\mu _{A,{\mathcal {B}}}\) is a spectral measure. In this paper, we investigate the spectral structure and spectral eigenmatrix problem associated with \(\mu _{A,{\mathcal {B}}}\) . Specifically, some sufficient and necessary conditions for a real diagonal matrix \(\Re \) to be a spectral eigenmatrix of \(\mu _{A,{\mathcal {B}}}\) are given under certain constraints on A and \(\Re \) . This extends some known results on the spectral eigenmatrix problem of planar self-affine measures.