<p>In unmanned aerial vehicle (UAV) power inspections, accurate segmentation of power line and transmission tower from aerial imagery is crucial for ensuring UAV flight safety. However, this task is challenged by a severe foreground–background class imbalance. To address this issue, this study proposes an adaptive feature enhancement network (AFENet), which adaptively enhances foreground features through pixel-level and feature-level information enrichment. AFENet employs a dual-branch architecture integrated with three newly designed modules: category-guided adaptive enhancement (CGAE) module, multi-shape convolution module (MSCM), and attention-guided enhancement module (AGEM). At the pixel level, the CGAE performs category-aware data augmentation based on the confidence scores of each class. At the feature level, the MSCM first refines local feature connectivity using a combination of strip, dilated, and square convolutions. Then, the AGEM leverages spatial attention and a newly designed multi-shape attention mechanism to strengthen the model’s capability in extracting target features under significant noise interference. Extensive experiments on a self-built dataset and two public datasets demonstrate the superiority and robustness of the proposed AFENet. Furthermore, the proposed AFENet has a parameter count of 6.87 MB, and even when the input image is resized to 2048 <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\times \)</EquationSource> <EquationSource Format="MATHML"><math> <mo>×</mo> </math></EquationSource> </InlineEquation> 2048 pixels, it still runs at 31.15 FPS on a single NVIDIA Geforce RTX-3060 GPU.</p>