Transformers rely on softmax attention, which scales quadratically with sequence length and can be computationally expensive for long sequences or high-resolution images. In this work, we investigate kernelized attention as a linear-time approximation to standard attention. We evaluate several kernel feature maps, including exponential, ELU, ReLU, polynomial, Tanh, and trigonometric mappings. On classification benchmarks, ELU, Tanh, and trigonometric kernels consistently achieve strong accuracy, with normalization further improving performance in most cases. These findings demonstrate that kernelized attention provides an effective and computationally efficient alternative to standard attention, with kernel choice and normalization playing critical roles in performance. This work opens the door to scalable transformer architectures that maintain accuracy while reducing computational overhead.