Effectively estimating numerous free parameters in large-scale biophysical neural network models based on sparse biological functional data is a formidable task. In this paper, we propose a simple yet efficient parameters optimization method called the iterative fine-grained genetic algorithm (IFGA) for rapid inference of the connection weights in the large-scale biophysical V1 models released by Allen Institute (referred to as the Allen V1 model) [2]. IFGA fully leverages the characteristics of the connectivity in Allen V1 model to reduce the dimensionality of parameters to be optimized. It also utilizes the relationship between optimization objectives and weight changes, as well as the discrepancies between model simulation data and expected data, to design an ingenious search strategy for each parameter, significantly improving the search efficiency in high-dimensional parameter space. A wealth of experimental results demonstrates that the proposed IFGA not only achieves better convergence of the Allen V1 model with the given mice data but also accomplishes faster convergence.