For \(s \in {\mathbb {C}}\) and \(0< a <1\) , let \(\zeta (s,a)\) and \(\mathrm{{Li}}_s (e^{2\pi ia})\) be the Hurwitz and periodic zeta functions, respectively. For \(0 < a \le 1/2\) , put \(Z(s,a):= \zeta (s,a) + \zeta (s,1-a)\) , \(P(s,a):= \mathrm{{Li}}_s (e^{2\pi ia}) + \mathrm{{Li}}_s (e^{2\pi i(1-a)})\) , \(Y(s,a):= \zeta (s,a) - \zeta (s,1-a)\) and \(O(s,a):= -i \bigl ( \mathrm{{Li}}_s (e^{2\pi ia}) - \mathrm{{Li}}_s (e^{2\pi i(1-a)}) \bigr )\) . Let \(n \ge 0\) be an integer and \(b:= r/q\) , where \(q>r>0\) are coprime integers. In this paper, we prove that the values \(Z(-n,b)\) , \(\pi ^{-2n-2} P(2n+2,b)\) , \(Y(-n,b)\) and \(\pi ^{-2n-1} O(2n+1,b)\) are rational numbers, in addition, \(\pi ^{-2n-2} Z(2n+2,b)\) , \(P(-n,b)\) , \(\pi ^{-2n-1} Y(2n+1,b)\) and \(O(-n,b)\) are polynomials of \(\cos (2\pi /q)\) and \(\sin (2\pi /q)\) with rational coefficients. Furthermore, we show that \(Z(-n,a)\) , \(\pi ^{-2n-2} P(2n+2,a)\) , \(Y(-n,a)\) and \(\pi ^{-2n-1} O(2n+1,a)\) are polynomials of \(0<a<1\) with rational coefficients, in addition, \(\pi ^{-2n-2} Z(2n+2,a)\) , \(P(-n,a)\) , \(\pi ^{-2n-1} Y(2n+1,a)\) and \(O(-n,a)\) are rational functions of \(\exp (2 \pi ia)\) with rational coefficients. Note that the rational numbers, polynomials and rational functions mentioned above are given explicitly. Moreover, we show that \(P(s,a) \equiv 0\) for all \( 0< a < 1/2\) if and only if s is a negative even integer. We also prove similar assertions for Z(s, a), Y(s, a), O(s, a) and so on. Furthermore, we prove that the function Z(s, |a|) appears as the spectral density of some stationary self-similar Gaussian distributions.