<p>Bilevel programming is one of the very active areas of research with many real-life applications in economics and engineering. Bilevel problems are hierarchical problems consisting of lower-level and upper-level problems, respectively. The leader or the decision-maker for the upper-level problem decides first, and then the follower or the lower-level decision-maker chooses his/her strategy. In the case of multiple lower-level solutions, the bilevel problems are not well defined, and there are many ways to handle such a situation. One standard way is to put restrictions on the lower level problems (like strict convexity) so that nonuniqueness does not arise. However, those restrictions are not viable in many situations. Therefore, there are two standard formulations, called pessimistic formulations and optimistic formulations of the upper-level problem (see Dempe (Foundations of Bilevel Programming. Nonconvex Optimization and its Applications. Kluwer Academic Publishers, Dordrecht, 2002)). A set-valued formulation has been proposed in the paper Dutta (in: Sollund, Stefes, Germani (eds) Generalized Nash Equilibrium Problems, Bilevel Programming and MPEC, Springer, Singapore, 2017) and has been studied in the literature (see Zemkoho (Set-Valued Var Anal 24:423-448, 2016); Pilecka (Set-valued optimization and its application to bilevel optimization. PhD thesis, TU Bergakademie Freiberg, Freiberg, Germany, 2015)). However, the study is limited to the continuous set-up with the assumption of value attainment, and the general case has not been considered. In this paper, we focus on the general case and study the connection among various notions of solution. Our main findings suggest that the set-valued formulation may not hold any bigger advantage than the existing optimistic and pessimistic formulation.</p>