The effective use of agricultural waste in sustainable construction is gaining attention, with rice straw offering potential as a fiber reinforcement for cement composites. However, its natural hydrophilicity and organic content limit bonding with the cement matrix. This study examines the effects of chemical and thermal treatments—sodium hydroxide (NaOH) at 1 M and 5 M concentrations, hydrochloric acid (HCl), and boiling—on rice straw fibers and their impact on fiber-reinforced cement composites. Fourier Transform Infrared (FTIR) spectroscopy and Contact angle (CA) measurements analyzed chemical modifications and surface wettability. Results showed that NaOH 1 M and boiling treatments improved fiber bonding by removing waxy substances and reducing hydrophobicity while preserving cellulose integrity. The contact angle decreased from 79.89° (untreated) to 41.43° (NaOH 5 M), enhancing wettability. However, NaOH 5 M caused excessive fiber degradation, reducing their mechanical properties in the fiber-reinforced cement composites. The findings highlight NaOH 1 M as the optimal treatment, balancing fiber modification, mechanical performance, and sustainability. This study supports the potential of rice straw as an eco-friendly reinforcement in cement composites, promoting sustainable construction materials.