Asymmetrical buildings with plan or vertical irregularity that are constructed in seismic prone areas are highly susceptible to damage during earthquake shaking. Architects often prefer to design buildings with vertical irregularity due to their aesthetics, but this can lead to serious safety concerns. Therefore, special attention must be given to the design of such irregular buildings, and various seismic codes may prohibit their construction in active seismic zones based on the degree of irregularity. The purpose of this study is to investigate the behavior of vertically irregular buildings with different types of irregularities, including mass, stiffness, and placement of shear walls. An ensemble of 19 building frames with various irregularities is considered and compared with a regular building frame. Static non-linear and time history analyses are carried out, and the results show that there is an increase in drift demand in the tower portion of set-back structures. The capacity of the structure decreases with the increase in irregularity.