Many complex optimization problems, for which standard methods do not provide good-enough solutions, require the utilization of efficient metaheuristics. However, it has been found that several metaheuristics suffer from different types of structural biases, that may undermine their performance. In this paper, we investigate one of the most used evolutionary computation methods, the Particle Swarm Optimization algorithm. We utilize a recently developed modular framework for the construction of 123,900 parameterizations of PSO and investigate the effect of the different parameter choices on the identified types of structural bias. We also analyze and discuss the impact of the different types of structural bias found in these parameterizations on their performance on functions from a standard test set.