Speech-to-text technology, a subfield of Automatic Speech Recognition, has significant advancements across various domains. Automatic Speech Recognition tools continue to encounter limitations in transcribing Arabic speech, particularly Modern Standard Arabic. These limitations stem from the language's rich morphology, diverse dialects, and the lack of high-quality datasets. This study investigates the effectiveness of commercial speech-to-text tools in transcribing Arabic speech. A novel dataset was created, comprising recordings from native Arabic speakers of varying ages, genders, and speech durations. Six transcription tools were evaluated using five similarity metrics. The experiments showed that Clipto yielded the best similarity score using the Cosine metric. The tool yielding the worst score, namely Notta, was investigated using Explainable AI. It was demonstrated that the topic used in the transcription, the talk speed, the speaker age, and the recording length significantly affect the tool’s transcription accuracy.