This article put forward a sparse Bayesian learning methodology incorporated with variational inference and an improved particle swarm optimization (PSO) algorithm for the first time to detect the structural damage of unsymmetrical frame structures. The variational Bayesian inference with delayed rejection adaptive Metropolis sampling is employed to solve the high dimensional integration of the posterior distributions of uncertain model parameters even the problem is unidentifiable. Compared to the conventional Bayesian model updating, the incorporation of the variational Bayesian inference with the improved PSO algorithm enhances the efficiency of model updating process. The methodology described was experimentally validated through an unsymmetrical frame in the laboratory. Model parameters as well as the hyperparameters were calculated via an iterative process so that the damage locations as well as the damage extents can be successfully identified. In addition, the involved uncertainties also can be evaluated by calculating the posterior distributions of the model parameters.