Efficient Extract, Transform, Load (ETL) pipelines are essential for processing large-scale datasets in today's data-centric environments. Traditional ETL methods, relying on static data partitioning and sequential computation, often suffer from poor scalability, high resource consumption, and extended execution times. To address these limitations, this paper proposes an AI-enabled ETL optimization framework that integrates machine learning-based dynamic partitioning with parallel processing techniques using Python libraries such as Dask, PySpark, and multiprocessing. Experimental validation, conducted using the TPC-H benchmark dataset, demonstrates that the AI-driven ETL pipeline achieves up to a 40% reduction in execution time and a 30% improvement in resource utilization compared to the baseline of traditional static-partitioned and sequential ETL processes. These findings highlight the potential of AI-driven optimization to significantly enhance the scalability, efficiency, and performance of ETL workflows for large and complex data processing applications.