Stone mastic asphalt (SMA) is a type of asphalt mix that has increased usage due to its ability to improve pavement serviceability. It is a gap-graded hot mix asphalt that contains a large percentage of coarse aggregate and bitumen filler mastic. These studies contribute to understanding SMA modification’s behavior and potential enhancements with steel slag and hydrated lime for pavement applications. Bamboo fiber was added to the mixture as cellular fiber to control drain-down. As we know, one of the primary causes of premature asphalt pavement failure is to start with a micro crack and progress to more complex damage. This study modified the SMA with steel slag and hydrated lime to enhance stability and rutting resistance. The rutting resistance was evaluated using the Hamburg Wheel Tracking Test method and compared with the Marshall stability test result. The study showed that substituting steel slag of aggregate and adding hydrated lime on SMA significantly enhanced rutting resistance and stability. Replacing aggregate size 4.75–9.5 mm with 60% of its weight in steel slag enhances rutting resistance by 20%. Adding hydrated lime improved rutting resistance from 15% to 33%.