Fine-tuning a pre-trained language model (PLM) on a specific downstream task is a well-established paradigm in natural language processing. However, training the entire model on downstream tasks is computationally expensive, requiring significant time and resources. Parameter efficient fine tuning (PEFT) has been proposed to address this challenge by reducing the number of trainable parameters. A popular category of PEFT techniques inserts trainable adapters in a frozen-parameter model during the fine-tuning stage. Common adapters include low-rank projections such as LoRA, which reduces the adapter’s representation power. We address this reduced representation using the Kronecker product instead of the low-rank projection to improve flexibility, leading to improved performance. We introduce KronA, a Kronecker equivalent of LoRA, to efficiently fine-tune Transformer-based PLMs. We apply different variants of KronA for fine-tuning the T5 model on the GLUE benchmark and show that KronA outperforms common PEFT baselines.