We study finite group actions on smooth manifolds of the form \(M\#\Sigma \) , where \(\Sigma \) is an exotic n-sphere and M is a closed aspherical space form. We give a classification result for free actions of finite groups on \(M\#\Sigma \) when M is 7-dimensional. We show that if \(\mathbb {Z}/p\mathbb {Z}\) acts freely on \(T^n\#\Sigma \) , then \(\Sigma \) is divisible by p in the group of homotopy spheres. When M is hyperbolic, we give examples \(M\#\Sigma \) that admit no nontrivial smooth action of a finite group, even though \({{\,\textrm{Isom}\,}}(M)\) is arbitrarily large. Our proofs combine geometric and topological rigidity results with smoothing theory and computations with the Atiyah–Hirzebruch spectral sequence.