Weed detection is an important research fields for precision agriculture, which is crucial to control the crop growing environments. Multiple machine learning algorithms have been applied to this field and achieve impressive results. However, the robustness of the detection has long been neglected. Traditional machine learning methods are usually unaware of the uncertainties in the prediction of weed detection. More specifically, they only know the categorical distribution over different classes as a point estimation. Knowing the uncertainty distribution can be important to perform weed detection and further agricultural decision-making because relying on an unreliable prediction can lead to unwanted and costly elimination of crops detected as weeds. To address this issue, an Uncertainty-Aware Robust Weed Detection with Evidential Neural Network is proposed to quantify the uncertainties of the detection process, which leads to more robust results. In the detection part, our methods used a transformer-based detection framework to increase performance and efficiency. Moreover, the transformer-based network makes it easier to apply our uncertainty quantification module. The uncertainty quantification module is an evidential neural network that can learn a Dirichlet distribution from the learned embedding space, which yields a sound estimation of the categorical estimation over a probability simplex. We conducted extensive quantitative experiments to illustrate the superior detection performance and sound uncertainty quantification, which showed the effective uncertainty calibration of our model. With model uncertainty, growers and automated robots can make more robust decisions knowing the uncertainty of each detection decision. That is, if the quantified uncertainty is large, we need to send the results to experts for further review, which reduces the chance of making overconfident yet unreliable decisions.