Early neonatal birth weight (BW) prediction is essential for mitigating low birth weight (LBW) risks and reducing fetal and neonatal morbidity and mortality. This study proposes a novel transformer-based bi-encoder model leveraging the TabNet architecture for early, interpretable neonatal BW prediction, using first-trimester maternal nutritional and health data. Unlike traditional methods that focus on late-pregnancy ultrasound scans, this model integrates overlooked early risk factors, such as nutritional deficiencies. Trained on an in-house dataset, the model achieved a mean absolute error (MAE) of 132 g and an R-squared (R \(^2\) ) of 0.9011, highlighting maternal folate and vitamin B12 levels as key determinants of BW. The approach offers improved early BW prediction, facilitating better prenatal care and clinical decision-making to reduce LBW risks.