In this paper, we are concerned with the nonlinear Helmholtz system of Hamiltonian type \(\begin{aligned} \left\{ \begin{array}{l} -\Delta u-k^2 u=P(x)|v|^{p-2}v,\quad \text {in}\ \mathbb {R}^N, \\ -\Delta v-k^2v=Q(x)|u|^{q-2}u,\quad \text {in}\ \mathbb {R}^N, \end{array} \right. \end{aligned}\) where \(N\ge 3\) , \(P,Q:\mathbb {R}^N\rightarrow \mathbb {R}\) are two positive continuous functions, the exponents \(p,q>2\) satisfy \(\frac{1}{p}+\frac{1}{q}>\frac{N-2}{N}\) . First, we obtained the existence of a ground state solution via a dual variational method. Moreover, the concentration behavior of such dual ground state solutions is established as \(k\rightarrow \infty \) , where a rescaling technique and the generalized Birman–Schwinger operator are involved. In addition, we also investigated the relation between the number of solutions and the topology of the set of the global maxima of the functions P and Q.