Modern methods primarily consider automatic exposure (AE) as an optimization problem based on metrics, which struggles with the challenge of defining a widely effective evaluation metric. In addition, the performance of the exposure algorithm is constrained by the mobile platform. Inspired by human perception, exposure preferences are greatly influenced by the environment and image content. We propose a novel and effective formulation for ultra fast exposure target estimation for smartphones’ AE control across all scenarios. Specifically, we approach the automatic exposure target estimation as a self-adaptive regression problem, utilizing multivariate multiscale histograms and prior information. Initially, we calculate the global histogram of Y-RGB channels and the local histogram of the Y channel across multiple scales, while simultaneously obtaining environmental prior information through inline sensors or preprocessing. We then determine the desired exposure target for the photograph through self-adaptive regression. Ultimately, convergence control is conducted, and an appropriate exposure setting is selected from the exposure table to achieve the targeted exposure level. Extensive experiments on our datasets and real smartphones demonstrate that our method excels in both stability and consistency. Our method significantly reduces computational costs by using statistical histogram inputs instead of 2D images. This ultra-light model can even reach over 550 FPS based on TensorFlow Lite on the low-end smartphone CPU (2 \(\times \) A75 2.2G/6 \(\times \) A55 1.8G). Our model has the remarkable features of low power consumption, high frame rate and ultra-lightweight, thus it can be deployed to smartphones with a wide range of performance.