Ancient Arabic manuscripts pose significant challenges to automatic recognition with varied calligraphic styles, ligatures, and document degradation. This work investigates hierarchical segmentation at four levels: lines, words, pseudo- words, and characters. A domain-specific dataset was assembled with various sources and annotated manually to reflect stylistic and structural variations. Images were preprocessed by normalization, denoising, and enhancement to enhance clarity. Three deep learning models—YOLOv8, Faster R-CNN, and Mask R-CNN—were tested with this dataset. Experiments indicate excellent performance by YOLOv8 for segmentation at the line level and superior character-level segmentation by Mask R-CNN. Faster R-CNN generates comparable performance at the word level with challenges with cursive scripts. These observations reflect a requirement for hybrid and transformer-based models to deal with fine-grained recognition tasks. The contributions to this work are releasing a hierarchical dataset and comparative benchmarking towards opening doors to sustainable OCR systems with a commitment to preserving Arabic manuscript heritages.