<p>The present work aims to analyze the hydrothermal behavior of multi-nanoparticle nanofluid flow (Cu–Al₂O₃/H₂O) within an octagonal cavity containing a cylinder with rectangular fins. Cylinders with rectangular fins are found in a large number of applications, viz. heat exchangers, automotive radiators, semiconductors, engines for cooling air, and hydrogen fuel cells. To enhance the heat transfer mechanism and thermal efficiency, the hybrid nanofluid is more efficient than the base fluid due to the introduction of various types of nanoadditives. The lower and top surfaces of an octagonal cavity are heated, whereas the vertical borders remain cooled with temperature, while the inclined borders are adiabatic. The internal heated cylinder with a radius <i>r</i> = 0.15<i>L</i>. It is presumed that the heated cylinder within the octagonal enclosure is surrounded by solid fins of variable sizes <i>l</i> = 0.07<i>L</i>, <i>l</i> = 0.12<i>L</i>, and <i>l</i> = 0.17<i>L</i>. The governing systems of equations are non-dimensionalized by incorporating suitable transformation. The resultant system is solved by applying the Galerkin finite element method in the computational COMSOL Multiphysics software. The study is focused on the influence of the radiation parameter (<i>R</i>), nanoadditives concentration (<i>ϕ</i>), angle of applied magnetic field (<i>α</i>) and size of fins (<i>H</i>) on the profiles of velocity (horizontal &amp; vertical), velocity magnitude, temperature, Nusselt number, Bejan number, and entropy generation via streamlines and isotherms are simulated. The findings reveal that the height of the fins and nanoadditives concentration play an important role in establishing and maintaining the temperature and heat transfer within the octagonal cavity. It is found that the angle of applied magnetic field significantly controls the entropy production and Bejan number.</p>