New cyclic groups based on the generalized order-k Pell sequences in the Heisenberg group and their application in cryptography

In this paper, we consider the finite groups \(\begin{aligned} H_{(t,l,m)}=\langle a,b,c | a^t=b^l=c^m=1, [a,b]=c, [a,c]=[b,c]=1\rangle . \end{aligned}\) $$\begin{array}{c}\hfill H_{(t,l,m)}=⟨a,b,c|a^t=b^l=c^m=1,[a,b]=c,[a,c]=[b,c]=1⟩.\end{array}$$ We obtain the generalized order-k Pell sequences and study their periods. We prove the period of the order-k Pell sequence divided the period of the generalized order-k Pell sequence in the Heisenberg group. Then, the generalized order-k Pell sequence in Heisenberg group are used to define new cyclic groups. As an application, these groups are used in encryption algorithms.