Recently, a growing interest has been seen in rapid medical image segmentation for point-of-care applications. UNeXt, a convolutional multilayer perceptron (MLP)-based rapid medical image segmentation network has shown an outstanding performance in single-organ segmentation. However, there is still a large room for improvement in multi-organ segmentation by exploring sufficient information from a global view. To this end, we propose UNeXt++, a more powerful framework that adds a lightweight Serial-Parallel Hybrid Attention module named SPHTension to the UNeXt. The proposed SPHTension is designed to assist in the detection and localization of lesion tissue by extracting image local features and global semantic context through parallel structures, respectively. These structures play distinct roles in instance segmentation. Furthermore, we introduce the Attentional Feature Fusion (AFF) approach, which simultaneously cascades learning blocks to fuse and optimize the feature representation. The proposed hybrid architecture is capable of simultaneously focusing on local and global features in different regions, effectively integrating them to sense the location and edges of lesion tissues, and performing accurate segmentation. It is noteworthy that UNeXt++ is capable of efficiently aggregating global representations by adding only a very small number of parameters. Experimental results demonstrate that our UNeXt++ outperforms UNeXt in terms of segmentation performance on the multi-organ segmentation dataset Synapse and three single-organ segmentation datasets. This improvement is observed to be between 5% and 18%, while the computational cost is reduced by 17% and the amount of parameters is reduced by 19%.