A biosensor based on surface plasmon resonance (SPR) with triangular indentations is proposed to detect and identify five types of cancer cells, exhibiting refractive indices ranging from \(\varvec{1.368}\) to \(\varvec{1.401}\) . This advanced biosensor features a multi-layer architecture comprising BK \(\varvec{7}\) /Ag/TiO \(_{\varvec{2}}\) /Al \(_{\varvec{2}}\) O \(_{\varvec{3}}\) grating and operates within the optical telecommunication band (OTB) ( \(\varvec{1260}\) – \(\varvec{1625}\) nm). To enhance the biosensor’s performance, HeLa cells are employed as reference cells, and various parameters—including the angle of light incidence, the thickness of the Ag and TiO \(_{\varvec{2}}\) layers, and grating specifications—are systematically analyzed to design a sensor with maximum sensitivity. The sensitivities achieved for the HeLa, Jurkat, PC- \(\varvec{12}\) , MDA-MB- \(\varvec{231}\) , and MCF- \(\varvec{7}\) cells are \(\varvec{9208.33}\) , \(\varvec{9714.28}\) , \(\varvec{10,857.14}\) , \(\varvec{11,785.71}\) , and \(\varvec{12,214.28}\) nm/RIU, respectively. Furthermore, the highest values for the sensor’s figure of merit (FoM) and quality factor (QF) are determined to be \(\varvec{122.14}\) /RIU and \(\varvec{14.53}\) . The highest FoM is recorded for MCF- \(\varvec{7}\) cell detection, while the maximum QF is observed for HeLa cell detection. Analysis of the electric field distribution under resonant conditions reveals that the sensor exhibits a strong electric field with a penetration depth of 500 nm. A comparative analysis of the proposed sensor against previous designs indicates a substantial enhancement in various performance parameters, including sensitivity, FoM, and QF.
