Probabilistic Neural Networks (PNNs), a category of Feedforward Neural Networks, leverage Kernel Density Estimators (KDEs) and the Bayesian conditional probability theorem for estimating conditional probabilities. Initially designed for classification, these networks exhibit commendable performance in both classification and regression tasks. The training process involves determining optimal or suboptimal values for the KDE smoothing parameter, commonly accomplished through analytical methods such as the Plug-in technique. Additionally, metaheuristic approaches like Particle Swarm Optimisation and Krill Herd Algorithm have been employed for smoothing parameter optimisation in PNNs due to the absence of gradient calculations. This contribution proposes the integration of Bacterial Foraging Optimisation (BFO) and Simulated Annealing (SA) for enhancing PNNs. The efficiency of these techniques in optimising PNNs is compared with the conventional Plug-in method, employing benchmark classification datasets sourced from UCI and Kaggle repositories. The results reveal that SA surpasses other methods in specific benchmarking tasks, suggesting its efficacy in training PNNs for specific problem domains.