Novel encryption method for color images based on cross-channel substitution and permutation using an improved two-dimensional Henon map (2D IHM)

This paper introduces symmetric chaos-based image encryption, incorporating innovative cross-channel substitution permutation techniques using an improved Henon map. Initially, we designed an improved Henon map (2D IHM) by integrating nonlinear transformations into the conventional Henon map, resulting in a broad and continuous range of control parameters and better randomness. The analysis of bifurcation diagrams, Lyapunov exponents, entropy, and NIST tests assure the improved chaotic characteristics such as ergodicity, sensitivity, and randomness of the 2D IHM and its suitability in encryption algorithms. 2D IHM generates the unpredictable random key streams through a novel key generation phase, utilizing an image-independent 512-bit secret key. These generated key streams are then employed in the encryption algorithm. The encryption algorithm initially calculates the hash value of the image using SHA 512; then, the hash values are modified using chaotic key streams via some nonlinear arithmetic operations and are randomly inserted as the last row of the plain image. Subsequently, this modified image undergoes inter-channel pixel substitution and permutation simultaneously. The proposed encryption process starts with substitution techniques that link key-based constants and previously enciphered pixels with current encrypting pixels through combined modulo addition and exclusive OR operations and performs novel cross-channel pixel permutations. Here, both operations are jointly controlled by the modified hash values and highly random chaotic key stream. The hash value imparts the image’s unique characteristics, and the hyperchaotic keystreams spread randomness into the encrypted image, ensuring unique and unpredictable encryption for each image even with the same encryption key. The evaluation metrics and approaches confirm that the proposed method ensures optimal statistical, differential, and randomness measures for cipher images. In addition, it also possesses vast keyspace with \(2^{512}\) $2^{512}$ -bit space, higher key sensitivity, resistance to brute force, cropping, noise, and known-chosen plaintext attacks. Furthermore, it exhibits improved time complexity compared to some recently proposed methods.