To conserve the cultural heritage in India, advanced deep learning models are needed with the ability to classify heritage sites images and at the same time be computationally efficient. ViTs have been incredibly successful in image classification but are typically computationally expensive, preventing their practical use. Using the Adaptive Logarithm Quantization (AdaLog) to enhance efficiency without performance degradation and thus optimizing Dual-ViT, a two-pathway transformer model, are presented in this work. AdaLog re-quantizes its logarithmic base based on discretization, and is therefore sensitive to power-law distributed activations, and loss of precision is reduced. In order to improve performance more, Fast Progressive Combining Search (FPCS) is applied to decide on the most appropriate quantization parameters, and the most appropriate choice of logarithmic base that guarantees effective compression with numerical stability. On the Indian Heritage Digital Space (IHDS) data, the AdaLog quantized Dual-ViT model has an accuracy of 85.68% (almost equal to the 87.63% of the original model) and requires only 29.88MB of model space (compared to 94.59MB). Also, computational complexity is reduced with a large reduction of 66.67% in FLOPS and hence this strategy is highly appropriate in the implementation on resource constrained edge devices. The above outcomes point to AdaLog enhanced Dual-ViT with FPCS as an effective and scalable heritage preservation system.