Contrastive Language-Image Pre-training (CLIP) has demonstrated strong performance across various downstream tasks. However, its text encoder is limited to a maximum of 77 tokens, making it incapable of handling longer textual inputs. Although some recent multimodal models have addressed this length limitation, they still struggle to capture structural semantics. As shown in Figs. 1 and 2, Long-CLIP struggle to distinguish between structurally correct and incorrect text descriptions based on the given image, as it tend to focus on learning general representation across varying text lengths while neglecting the modeling of structured representation. In this paper, We propose MP-CLIP, a fine-tuned CLIP variant that supports longer inputs and enhances structured representation for better understanding of both short and long descriptions. MP-CLIP introduces two key components: (1) Multi-Segment Aggregation Encoder, and (2) Multi-Granularity Matching strategy. MP-CLIP achieves an average 3% improvement over Long-CLIP on long-text image retrieval tasks, and surpasses CLIP on short-text retrieval benchmarks such as MSCOCO and Flickr30k. It also maintains comparable performance to CLIP on zero-shot image classification, and achieves superior results on fine-grained tasks, outperforming Long-CLIP by 7% on VG-Relation. Moreover, MP-CLIP can be seamlessly integrated into image generation pipelines to guide high-quality image synthesis from detailed descriptions.