Ultrasound nerve segmentation is a crucial task in medical imaging as it contributes to accurate diagnosis and treatments of various nerve disorders. Deep-learning based image segmentation continues to pose a challenge especially on the techniques to handle low contrast images and noise in nerve ultrasound as well as limited dataset availability to perform deep-learning. Besides, performing manual nerve segmentation is subjected to interobserver variability and time-consuming. In recent years, U-net has been ubiquitous in ultrasound image segmentation for vital localization of nerve structures. However, U-net architecture has the restriction of having limited receptive field in encoder and decoder which leads to inefficient use of contextual information and misclassification of local features. Therefore, this study focused on a novel type U-Net variant which utilized U-net as the backbone architecture and incorporated with attention model at the decoder to leverage its advantage in exploring spatial focus on more relevant parts in the input images of brachial plexus nerves. To further diminish irrelevant feature maps especially on smaller annotated dataset that is prone to vanishing gradient issue, different loss functions and data augmentation methods were employed to train the dataset to promote the model in learning more distinctive features. Dice Coefficient and Intersection Over Union (IoU) were implemented as the metrics to evaluate the Attention U-Net model with base U-Net model. The improved Attention U-Net with Relu-6 as the activation function in attention mechanism was proven to improve the inference time by 9 times and the performance in dice coefficient by 1.19 times.